Overview
Copilot's usefulness lives or dies on whether it actually does what people need reliably, safely, and across an enormous range of real enterprise and consumer scenarios. We're looking for a Principal Product Manager to own how we measure that.
You'll build and run the evaluation systems that determine whether a model update, a new Copilot skill, or an agentic workflow is ready to ship. You'll partner directly with applied research, model training, and engineering teams to turn "does this feel better" into concrete, defensible metrics and you'll use those metrics to shape what we prioritize next.
This is a technical, judgment-heavy PM role. You won't just write requirements docs; you'll personally dig into transcripts, build eval sets, argue about what "good" means for a given surface, and make the call on whether something is ready for customers.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
- Own the eval lifecycle end-to-end: design, build, and maintain evaluation suites (offline benchmarks, live A/B experiments, agentic task suites, red-team/adversarial sets) that measure quality, safety, and task success across Copilot surfaces (Microsoft 365, Windows, Edge, Copilot Studio agents, etc.)
- Gate model and feature launches: define entry/exit criteria for shipping model updates or new capabilities, and make the ship/no-ship call in partnership with research and engineering leadership
- Translate model behavior into product requirements: read outputs and failure modes directly, identify patterns, and turn them into prioritized fixes — whether that's a training data gap, a prompt/system-message change, or a UX guardrail
- Build feedback loops: stand up pipelines that turn user feedback, support signals, and enterprise customer escalations into structured eval cases, so regressions get caught before they reach customers
- Partner cross-functionally: work daily with applied scientists, ML engineers, Responsible AI, and Trust & Safety teams to represent user needs in model development, and represent model capabilities/limitations back to the broader product org
- Define what "good" looks like for ambiguous, high-stakes scenarios (enterprise data handling, agentic actions taken on a user's behalf, multi-turn reasoning) where there's no existing playbook
- Communicate quality tradeoffs clearly to leadership — what shipped, what didn't, what the residual risk is, and why.
Qualifications
**Required Qualifications: **
Bachelor's Degree AND 8+ years experience in product/service/program management or software development
- OR equivalent experience.
Preferred Qualifications:
Bachelor's Degree AND 12+ years experience in product/service/program management or software development
- OR equivalent experience.
4+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework).
6+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn).
6+ years experience disrupting a market for a product, feature, or experience (e.g., competitive disruption, taking the place of an established competing product).
Product Management IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about **requesting accommodations.**