AI Product Manager

EvenUp EvenUp · Vertical AI · San Francisco, CA · Hybrid · Product Management

AI Product Manager to lead the development of core models for entity extraction and linking in legal and medical domains. Responsibilities include roadmap ownership, prompt engineering, building evaluation frameworks, managing the end-to-end model lifecycle, balancing technical trade-offs, and researching emerging techniques. Requires 5+ years of experience in AI modeling or data science products, with deep understanding of LLMs, evals, RAG, agentic orchestration, and ML.

What you'd actually do

  1. Own the strategic roadmap and tactical execution for core models, focusing on domain-specific entity extraction and linking models in the legal and medical spaces.
  2. Dive deep into prompt engineering; build robust evaluation frameworks and gold-standard datasets to measure model performance and reasoning accuracy.
  3. Partner with ML engineers, data scientists, software engineers, and legal/medical subject matter experts to manage the end-to-end model lifecycle from data curation to post-production monitoring.
  4. Balance technical trade-offs between model size and reasoning quality to ensure our features are both intelligent and commercially viable.
  5. Research and experiment with emerging modeling techniques to maintain our competitive edge.

Skills

Required

  • 5+ years of professional experience with a heavy emphasis on high-trajectory growth in AI modeling or data science products
  • Deep understanding of LLMs, evals, RAG, agentic orchestration, classical machine learning, and statistics
  • Proven track record of treating data quality as a first-class product and building pipelines designed specifically for model training
  • Demonstrated ability to translate complex model metrics into clear business impact for non-technical stakeholders

Nice to have

  • prompt engineering
  • building robust evaluation frameworks
  • gold-standard datasets
  • post-production monitoring
  • technical trade-offs between model size and reasoning quality
  • emerging modeling techniques

What the JD emphasized

  • core models
  • training, optimizing, and evaluating numerous models
  • increasing accuracy
  • building new models
  • building data flywheels
  • state of the art in foundation models and agentic orchestration
  • models
  • grinding on model accuracy
  • AI modeling or data science products
  • LLMs, evals, RAG, agentic orchestration
  • model training
  • model metrics

Other signals

  • training, optimizing, and evaluating numerous models
  • increasing accuracy across key entity extraction models
  • building new models
  • building data flywheels
  • state of the art in foundation models and agentic orchestration