Research Scientist - Frontier Benchmarks

Snorkel AI Snorkel AI · Data AI · Redwood City, CA +1 · Remote · 316 - Research

Research Scientist focused on developing frontier benchmarks and datasets for LLM training and evaluation, collaborating with academic partners and customers, and influencing company roadmap and research community.

What you'd actually do

  1. Design state of the art datasets that drive frontier model training and evaluation based on current model performance and academic partnerships
  2. Translate benchmark insights into clear, compelling narratives that articulate the ROI of expert-curated data for customer-facing presentations, technical reports, and go-to-market materials.
  3. Work cross-functionally with data operations, product, engineering, and strategy to surface research findings that inform the company roadmap.
  4. Stay at the frontier of LLM evaluation research and bring best practices into Snorkel's workflows
  5. Represent Snorkel's research externally through publications, blog posts, conference talks, and customer engagements that advance the conversation around data-centric AI

Skills

Required

  • AI/ML evaluation
  • NLP
  • rigorous experimental design
  • measuring the impact of training and evaluation data on model behavior
  • communication skills
  • present complex technical findings clearly to both technical and non-technical audiences
  • operating in a fast-moving, cross-functional environment with ambiguous problem spaces

Nice to have

  • Ph.D. in machine learning, NLP, or a related field
  • GTM strategy
  • startup dynamics
  • commercial side of AI data services

What the JD emphasized

  • track record of rigorous experimental design
  • frontier model training
  • LLM evaluation research

Other signals

  • design datasets
  • frontier model training
  • LLM evaluation research
  • data-centric AI