Product Manager, AI Models

Descript Descript · AI Frontier · San Francisco, CA · Product & Design

Product Manager to lead the AI Research and Enablement roadmap, focusing on integrating cutting-edge AI research with production ML infrastructure and product strategy. Responsibilities include making build vs. buy decisions, guiding research direction, owning the evals strategy, driving quality standards for AI models, and optimizing costs and infrastructure for AI capabilities. Requires experience in AI/ML product management, understanding of ML systems and LLMs, and experience with production AI/ML shipping, evals frameworks, training, and inference infrastructure.

What you'd actually do

  1. Make build vs. buy decisions: Evaluate when to train our own models vs. integrate third-party solutions based on market gaps, competitive advantage, and ROI
  2. Own the evals strategy: Design evaluation frameworks that are productionized and tied to real user needs, not just academic metrics
  3. Partner with product teams: Advise on which models or architectures are best suited for specific features over time
  4. Optimize COGS: Make strategic decisions on model selection, caching strategies, and infrastructure to balance quality, latency, and cost
  5. Guide research direction: Use product insight to inform what the team trains and develops; use research understanding to guide product direction

Skills

Required

  • 4+ years of product management experience
  • Understanding of modern ML/AI systems and LLMs
  • Experience shipping AI/ML products to production at scale
  • Experience with evals frameworks, model training pipelines, and inference infrastructure
  • Understanding of ML cost structures (training compute, inference costs, token economics)
  • Experience working with research teams
  • Track record of partnering with engineering teams on infrastructure and platform work
  • Comfortable operating in ambiguity and setting direction when the path isn't clear

Nice to have

  • Experience with SQL, experimentation platforms, and analytics tools

What the JD emphasized

  • at least 1-2 years working on AI/ML products
  • Track record of making sound build vs. buy decisions in the AI space
  • Experience balancing research exploration with shipping product value
  • shipping AI/ML products to production at scale
  • Experience with evals frameworks, model training pipelines, and inference infrastructure

Other signals

  • product strategy
  • AI capabilities
  • AI Research
  • LLM Research
  • AI Enablement
  • MLOps infrastructure
  • evals strategy
  • quality standards
  • model performance
  • AI features
  • research infrastructure
  • training infra
  • inference infra
  • data pipelines