Staff Machine Learning Engineer, Merchants

Pinterest Pinterest · Consumer · Toronto, ON · Monetization

Staff Machine Learning Engineer focused on building AI/ML systems, including LLMs, to improve merchant presence and shopping experiences on Pinterest. The role involves leading LLM-first, evaluation-driven initiatives for agentic workflows, measurement, and operational rigor, as well as advancing core relevance capabilities. Responsibilities include end-to-end technical delivery, setting technical direction, building ML/GenAI systems for merchant quality, establishing evaluation practices, designing for quality/cost/latency/reliability/safety, defining the ML engineering operating model, partnering with stakeholders, driving experimentation, mentoring, and supporting hiring.

What you'd actually do

  1. Own end-to-end technical delivery for cross-team initiatives—from problem framing and technical strategy through architecture, implementation, rollout, monitoring, and iteration.
  2. Set technical direction and execution plans in partnership with a Director and cross-functional leads, including defining milestones, sequencing, and quality bars for the domain.
  3. Build and evolve ML and GenAI systems that improve merchant quality and understanding (e.g., merchant content enrichment, attribute extraction/normalization, entity resolution, merchant/brand quality signals, and policy-aware transformations), with clear downstream impact on retrieval, ranking, and shopping surfaces.
  4. Establish robust evaluation and measurement practices across ML + LLM-assisted systems, including golden datasets, human-in-the-loop review loops, automated regression testing, offline/online metric alignment, and clear go/no-go launch criteria for quality, safety, and performance.
  5. Design systems with strong attention to quality, cost, latency, reliability, and safety, including guardrails, fallbacks, caching, and observability to support scaled production operations.

Skills

Required

  • 8+ years of industry experience in ML engineering / applied ML / software engineering
  • Staff-level (or equivalent) IC delivering complex production systems
  • Demonstrated ability to lead 0→1 ML/LLM efforts
  • Strong track record shipping ML-powered systems in domains such as recommendation, ranking, retrieval, content understanding, ads relevance, commerce, or adjacent areas with clear product impact
  • Hands-on experience building LLM-powered applications in production (or adjacent GenAI systems)
  • Deep experience with evaluation and measurement: dataset strategy, labeling/review operations, metric design, regression testing, and connecting offline improvements to online outcomes
  • Strong systems design skills building data- and ML-intensive systems

Nice to have

  • Experience with agentic workflows
  • Experience with measurement and operational rigor
  • Experience with merchant integrity and business integrity
  • Experience with core relevance capabilities such as merchant/brand affinity modeling
  • Experience with shopping discovery
  • Experience with guardrails, fallbacks, caching, and observability
  • Experience with ML platform teams
  • Experience with Trust/Policy/Legal stakeholders
  • Experience with experimentation (A/B tests, holdouts)
  • Experience with error analysis
  • Experience mentoring and raising the bar for technical design, evaluation rigor, and production readiness
  • Experience supporting hiring and onboarding

What the JD emphasized

  • lead LLM-first, evaluation-driven initiatives
  • agentic workflows
  • measurement
  • operational rigor
  • merchant integrity
  • business integrity
  • advance core relevance capabilities
  • merchant/brand affinity modeling
  • shopping discovery
  • technical lead for ML
  • define the technical roadmap
  • establish engineering standards
  • scaling the domain and team
  • user trust
  • relevance
  • shopping outcomes
  • high-traffic Pinterest surfaces
  • organic and paid
  • 8+ years of industry experience
  • Staff-level (or equivalent) IC
  • complex production systems
  • lead 0→1 ML/LLM efforts
  • ambiguous problem spaces
  • delivering a production system with measurable impact
  • shipping ML-powered systems
  • recommendation, ranking, retrieval, content understanding, ads relevance, commerce, or adjacent areas
  • clear product impact
  • Hands-on experience building LLM-powered applications in production
  • strong judgment on reliability, failure modes, rollout safety, and practical tradeoffs
  • Deep experience with evaluation and measurement
  • dataset strategy
  • labeling/review operations
  • metric design
  • regression testing
  • connecting offline improvements to online outcomes
  • Strong systems design skills
  • data- and ML-intensive systems
  • navigate tradeoffs in performance, r

Other signals

  • LLM-first initiatives
  • agentic workflows
  • evaluation-driven
  • merchant integrity
  • business integrity
  • relevance capabilities
  • shopping discovery
  • technical lead for ML
  • define technical roadmap
  • establish engineering standards
  • scaling the domain and team
  • user trust
  • relevance
  • shopping outcomes
  • high-traffic Pinterest surfaces
  • organic and paid