Technical Program Manager, Gemini Evals, Deepmind

Google Google · Big Tech · Mountain View, CA +1

Technical Program Manager at Google DeepMind focused on Gemini Evals. The role involves collaborating with engineering and data science teams to design, integrate, and execute model evaluations, conduct loss analysis, and drive strategic goals for AI programs. Requires experience in leading engineering projects and understanding LLM evals, model training, or data science.

What you'd actually do

  1. Collaborate closely with the engineering team to support infrastructure development, meticulously track priorities, issues, deeply understand evaluation and experimentation results, coordinate alignment and, assessment with model teams.
  2. Collaborate with data scientists to design and integrate model evals, execute evals, and conduct loss analysis to inform model quality.
  3. Drive strategic goals and tactical delivery. Lead with high agency. Proactively identify workflow gaps and implement scalable improvements to optimize efficiency and output quality.

Skills

Required

  • Bachelor's degree in Computer Science, Engineering, Data Science, related technical field, or equivalent practical experience.
  • 5 years of experience leading engineering projects across multiple geographies and time zones.

Nice to have

  • Experience in people management.
  • Experience in Large Language Model (LLM) evals, model training, model releases, or data science.
  • Ability to quickly learn and deeply understand the technical aspects of the programs from interface to infrastructure, serving, and customer issues, and drive technical discussions.
  • Excellent skills in managing complex stakeholder relationships.

What the JD emphasized

  • deeply understand evaluation and experimentation results
  • design and integrate model evals
  • execute evals
  • Large Language Model (LLM) evals

Other signals

  • managing AI projects
  • evaluating AI models
  • driving AI product innovation