Strategic Projects Lead, Safety

Handshake Handshake · Enterprise · Seattle, WA · HAI Delivery Ops

The Strategic Projects Lead, Safety role at Handshake AI focuses on executing large-scale human data programs for frontier AI model training and evaluation. This involves designing and managing annotation frameworks, leading annotator teams, developing annotation playbooks, and collaborating with AI labs to inform model policy, training, and evaluation decisions. The role also involves detecting data trends and improving safety policies. It is a high-ownership, outcomes-driven position requiring strong analytical and operational scaling skills.

What you'd actually do

  1. Design and architect annotation frameworks and labeling workflows that scale effectively while maintaining high precision, accuracy, and inter-rater agreement
  2. Lead, mentor, and manage a team of trusted annotators, fostering a culture of precision, consistency, and care given the sensitive nature of the work while addressing complex edge cases, and identify policy inconsistencies
  3. Collaborate with policy leads to offer organized insights regarding gaps, classification ambiguities, and nuanced scenarios derived from practical data annotation
  4. Construct and iterate on detailed annotation playbooks and auditor resources to drive precision and alignment across diverse datasets and regulatory domains
  5. Detect and surface novel data trends, systemic labeling errors, and strategic opportunities to harden and mature internal safety policies

Skills

Required

  • 2+ years of experience in trust and safety, policy enforcement, regulatory compliance, regulatory compliance, risk management, management consulting, government or related field
  • Strong analytical and first-principles problem-solving skills applied to scaling operations
  • proficiency in SQL (or similar) to draw insights from large data sets
  • Exceptional communication and stakeholder management skills, including with senior customers
  • High ownership mindset with pride in end-to-end accountability
  • Curiosity and ability to quickly learn technical AI concepts and industry trends

Nice to have

  • Experience in scaling policy enforcement or content review workflows across large, distributed workforces
  • Subject matter expertise in high severity policy harms, such as violent activities, dangerous organizations / individuals, or sexual activities
  • Experience developing evaluations for harmful content or policy enforcement use-cases

What the JD emphasized

  • model policy annotation
  • evaluations
  • policy annotation
  • regulatory domains

Other signals

  • human data for AI training
  • frontier AI labs
  • model policy annotation
  • evaluations for AI systems