Senior Manager, Forward Deployed Research

Snorkel AI Snorkel AI · Data AI · Redwood City, CA +1 · Remote · 415 - DaaS Sales & Success

Senior Manager, Forward Deployed Research at Snorkel AI. This role focuses on benchmarking frontier AI models and tuning them on proprietary data to demonstrate the value of Snorkel's data services. It involves defining methodologies, building tooling, and leading a small team of engineers and researchers. The role requires strong applied ML, model evaluation, and leadership skills, with a focus on translating technical results into GTM insights.

What you'd actually do

  1. Own the system for measuring what our data does: define how we benchmark and tune models on our data, and the tooling and automation the function needs, partnering with Engineering to build it
  2. Recruit, hire, and develop a small team of engineers and researchers; a player-coach role, hands-on technical work plus team leadership
  3. Own the methodology and playbook for benchmarking and tuning: the model panels, the metrics we report, the tuning setups, and the quality bar, so results are consistent, repeatable, and defensible across accounts and data series
  4. Turn benchmark results into gap intelligence: clear analyses of where models fall short that serve as the evidence behind our data pitch and a primary input to what we build next
  5. Tune customer and open-source models on our data to demonstrate the lift it produces, and turn that into presales material and intelligence

Skills

Required

  • Applied ML
  • Model evaluation
  • Research-intensive engineering
  • Building technical systems
  • Software engineering
  • Building data or evaluation pipelines
  • Automation
  • Tooling
  • Model benchmarking
  • Designing evaluations
  • Selecting model panels and metrics
  • Producing rigorous, defensible results
  • LLMs
  • Evaluation methodologies
  • Post-training techniques (RLHF, DPO, RLAIF)
  • Coding agents
  • Reasoning
  • Multimodal models
  • RL environments
  • Leading technical talent
  • Hiring technical talent
  • Developing technical talent
  • Translating technical results into clear insights
  • Partnering effectively across research, engineering, and GTM
  • Working in a fast-moving environment
  • Comfortable with ambiguity
  • Rapid iteration

Nice to have

  • M.S. in Computer Science, Machine Learning, or related field

What the JD emphasized

  • player-coach
  • hands-on technical work
  • team leadership
  • evaluation depth
  • hold their own on frontier AI
  • own a function
  • grow a team
  • building data or evaluation pipelines, automation, and tooling that scale
  • Deep understanding of model evaluation and benchmarking
  • Strong fluency in frontier AI concepts
  • Experience leading, hiring, and developing technical talent

Other signals

  • benchmarking frontier models
  • tuning models on proprietary data
  • demonstrating model lift
  • player-coach role
  • leading a team