Senior AI Forward Deployed Engineer

Handshake Handshake · Enterprise · San Francisco, CA · Engineering

Senior Forward Deployed AI Engineer role at Handshake AI, focusing on partnering with AI labs to translate research requirements into evaluation frameworks, prototype pipelines, and tooling. The role involves owning the full lifecycle of research engagements, making design decisions, mentoring engineers, and staying current on AI advancements. Requires strong Python, ML stack experience, knowledge of reinforcement learning and post-training techniques, and hands-on fine-tuning experience.

What you'd actually do

  1. Partner directly with AI lab researchers to understand their post-training goals and data requirements, translating ambiguous research questions into scoped, executable projects
  2. Design and deliver evaluation frameworks, annotation pipelines, and benchmark infrastructure tailored to each lab's training methodology
  3. Prototype and iterate fast: stand up lightweight experiments, run evals, and interpret results in tight feedback loops with research partners
  4. Make key design decisions around data quality and evaluation design that hold up at scale
  5. Mentor and uplevel other engineers and researchers on the team, establishing technical standards for forward-deployed AI work

Skills

Required

  • Python
  • ML stack
  • Data processing
  • Model evaluation
  • Experiment tracking
  • Pipeline tooling
  • Reinforcement learning
  • Post-training concepts (RLHF, DPO, PPO)
  • Fine-tuning
  • Lightweight optimization of ML models (Tinker, LoRA, PEFT)
  • ML data pipelines
  • Data labeling systems
  • Eval frameworks
  • Quality metrics
  • Communication
  • Stakeholder management
  • Prioritization
  • Leading technical projects

Nice to have

  • Evaluation design for LLMs or RLHF pipelines in production customer environments
  • Published research or benchmarking work
  • Contributions to open-source AI/ML tooling
  • Forward-deployed, solutions engineering, or technical consulting role at a high-growth AI company
  • Annotation platform tooling
  • Quality control frameworks
  • Human feedback collection at scale

What the JD emphasized

  • 6+ years of experience in applied ML, AI research engineering, or a closely related field with real exposure to model training workflows and post-training techniques
  • Solid working knowledge of reinforcement learning and post-training concepts (RLHF, DPO, PPO, etc.). You don't need to have trained frontier models, but you need to hold your own in a room of people who have
  • Hands-on experience fine-tuning or lightweight optimization of ML models (Tinker, LoRA, PEFT, or similar). You've actually tinkered with models, not just read about it

Other signals

  • Partnering with frontier AI labs
  • Translating ambiguous research requirements into concrete evaluation frameworks
  • Prototyping pipelines and tooling
  • Deep AI knowledge and customer ownership
  • Influencing how frontier models get trained