Senior Research Scientist, Gemini Release Evaluations, Deepmind

Google Google · Big Tech · Mountain View, CA +2

Research Scientist focused on evaluating and improving AI models, specifically Gemini, by creating new datasets and understanding the relationship between static evaluations and live production traffic. The role involves authoring research papers and driving project work related to AI model training, testing, evaluation, and tuning.

What you'd actually do

  1. Author research papers to share and generate impact of research results across function and in the research community.
  2. Drive project work by defining the data structure, framework, design, and evaluation metrics for research solution development and implementation. Identify timelines and obtain resources needed.
  3. Identify gaps in our existing release evals and create new SOTA datasets that push frontier models to their limits.
  4. Help build a deeper understanding of the relationship between static evaluations and live production traffic.

Skills

Required

  • AI model training
  • AI model testing
  • AI model evaluation
  • AI model tuning
  • LLM release cycles
  • timeline management
  • data structure definition
  • framework design
  • evaluation metrics design
  • research agenda leadership

Nice to have

  • coding experience
  • research efforts leadership
  • influencing other researchers

What the JD emphasized

  • PhD in Computer Science, a related field, or equivalent practical experience.
  • 2 years of experience leading a research agenda.
  • 1 year of experience in a data science field.
  • Experience with AI model training, testing, evaluation, and tuning processes, as well as LLM release cycles and timeline management.

Other signals

  • research papers
  • evaluation metrics
  • frontier models
  • static evaluations vs live production traffic