Research Scientist, Safety Post Training

Scale AI Scale AI · Data AI · San Francisco, CA · Research

Research Scientist focused on developing and applying post-training methods and interpretability techniques to enhance the safety, robustness, and alignment of frontier AI systems. The role involves designing post-training pipelines, conducting evaluations to understand model behaviors, and collaborating to translate findings into safety standards and best practices.

What you'd actually do

  1. Design and run post-training pipelines to study how training choices affect model safety, robustness, and alignment properties
  2. Develop interpretability-informed evaluations that reveal how and why models produce unsafe, deceptive, or otherwise undesirable behaviors, and use those insights to guide targeted mitigations
  3. Collaborate with policymakers, engineers, and other researchers to translate post-training and interpretability findings into actionable safety standards, evaluation benchmarks, and best practices

Skills

Required

  • post-training methods
  • interpretability techniques
  • RLHF
  • DPO
  • GRPO
  • published research in machine learning
  • generative AI
  • sophisticated ML problems
  • written and verbal communication skills

Nice to have

  • mechanistic interpretability
  • probing
  • understanding model internals
  • red-teaming
  • adversarial evaluation
  • failure modes introduced or masked by post-training
  • reward hacking
  • sycophancy
  • alignment faking

What the JD emphasized

  • post-training methods
  • interpretability techniques
  • post-training
  • interpretability
  • safety
  • robustness
  • alignment

Other signals

  • develop and apply post-training methods
  • interpretability techniques to make frontier AI systems safer
  • study how training choices affect model safety, robustness, and alignment properties
  • translate post-training and interpretability findings into actionable safety standards