Principal Data Scientist

Microsoft Microsoft · Big Tech · Redmond, WA +4 · Technical Support Engineering

Principal Data Scientist role focused on designing and analyzing large-scale experiments and observational studies, developing and productionizing advanced machine learning and causal models, and establishing best practices for experimentation and model validation. The role also involves evaluating LLMs and AI agents, building evaluation frameworks, and influencing metric definitions and roadmaps through data-driven insights. This role is critical for ensuring the quality, safety, and reliability of AI systems within the Customer Service & Support organization.

What you'd actually do

  1. Lead the design and analysis of large-scale experiments and observational studies; select and defend the right causal inference approach for each problem.
  2. Develop and productionize advanced machine learning and causal models that drive strategic decisions.
  3. Establish best practices for experimentation, statistical rigor, and model validation across the team.
  4. Influence metric definitions, measurement strategy, and roadmap through data-driven insight.
  5. Mentor Associate data scientists and review analyses for methodological soundness.

Skills

Required

  • Bachelor's Degree in Computer Science, Information Technology, or related field AND 10+ years technical support, technical consulting experience, or information technology experience
  • Ability to meet Microsoft, customer and/or government security screening requirements
  • Citizenship & Citizenship Verification: This position requires verification of U.S. citizenship

Nice to have

  • Bachelor's Degree in Computer Science, Information Technology, or related field AND 15+ years of technical support, technical consulting experience, or information technology experience
  • Expert-level causal inference — able to choose, apply, and critique experimental and observational methods (DiD, IV, propensity/matching, uplift, synthetic control).
  • Advanced statistics — deep command of inference, uncertainty quantification, and both Bayesian and frequentist approaches.
  • Strong ML expertise — end-to-end modeling, robust validation, and production deployment.
  • Experience evaluating LLMs and/or AI agents — building rigorous evaluation frameworks, benchmarking, and measuring quality, safety, and reliability at scale.
  • Track record leading impactful experimentation programs.
  • Experience with causal ML libraries (DoWhy, EconML, CausalML) at scale.
  • Demonstrated technical leadership and mentorship.
  • Data storytelling — translating complex analyses into compelling data stories through enhanced visualizations in Python (e.g., matplotlib, seaborn, plotly) or other tools.
  • Expert proficiency in Python or R and SQL.

What the JD emphasized

  • Expert-level causal inference
  • Advanced statistics
  • Strong ML expertise
  • Experience evaluating LLMs and/or AI agents
  • Track record leading impactful experimentation programs
  • Expert proficiency in Python or R and SQL

Other signals

  • productionize advanced machine learning and causal models
  • establish best practices for experimentation, statistical rigor, and model validation
  • influence metric definitions, measurement strategy, and roadmap through data-driven insight
  • experience evaluating LLMs and/or AI agents
  • building rigorous evaluation frameworks, benchmarking, and measuring quality, safety, and reliability at scale