Principal Software Engineer

Microsoft Microsoft · Big Tech · Redmond, WA +1 · Software Engineering

This role focuses on building and enhancing the evaluation platform for M365's AI offerings, enabling builders to run faster and more comprehensive evaluations throughout the development lifecycle. The goal is to automate tasks and improve performance understanding of AI features.

What you'd actually do

  1. Partners with appropriate stakeholders to determine user requirements for a set of scenarios.
  2. Leads identification of dependencies and the development of design documents for a product, application, service, or platform.
  3. Leads by example and mentors others to produce extensible and maintainable code used across products.
  4. Leverages subject-matter expertise of cross-product features with appropriate stakeholders (e.g., project managers) to drive multiple group's project plans, release plans, and work items.
  5. Holds accountability as a Designated Responsible Individual (DRI), mentoring engineers across products/solutions, working on-call to monitor system/product/service for degradation, downtime, or interruptions.

Skills

Required

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • Ability to meet Microsoft, customer and/or government security screening requirements

Nice to have

  • Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • Demonstrated technical leadership experience in evaluation, distributed systems or development platforms.
  • Experience building systems to evaluate and drive quality in a product and using data to drive engineering decisions.
  • Solid platform engineering mindset: building reusable components, reducing time‑to‑launch, improving debuggability, and delivering well‑documented tooling.

What the JD emphasized

  • evaluation system
  • evaluation
  • evals
  • eval

Other signals

  • evaluation system
  • AI offerings
  • user journeys of an eval system
  • fine tuning a model
  • launching a new feature or experiment
  • adding metrics
  • onboarding a new 1P or 3P partner
  • understanding user feedback
  • creating query sets