Software Engineer II

Microsoft Microsoft · Big Tech · Redmond, WA +1 · Software Engineering

Full Stack Software Engineer to build capabilities for Microsoft Copilot, working across evaluation platform stages: data sampling, AI data collection/processing, data analysis/evaluation, and insight generation. Responsibilities include experimentation, pipeline design, data analysis, and dashboard creation, with collaboration across M365 teams to assess Copilot's performance. Familiarity with LLMs, prompt engineering, and cloud infrastructure is preferred.

What you'd actually do

  1. Expertise in experimentation methodologies, including A/B evaluation, data sampling, measurement techniques, evaluation design, and data analysis.
  2. Demonstrating strategic vision by understanding organizational goals, translating metrics into actionable insights, and enhancing product quality.
  3. Designing pipeline architecture to ensure rapid iteration and scalability.
  4. Conducting post-analysis of labeled data and developing dashboards to visualize insights.
  5. Collaborating closely with the product team to enhance quality and address gaps.

Skills

Required

  • Bachelor's Degree in Computer Science or related technical field AND 2+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • Ability to meet Microsoft, customer and/or government security screening requirements

Nice to have

  • Master's Degree in Computer Science or related technical field AND 3+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • Bachelor's Degree in Computer Science or related technical field AND 5+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
  • 2+ years’ experience with SQL, PostgreSQL or MySQL.
  • 2+ years' experience building scalable services on top of public cloud infrastructure like Azure, AWS, or GCP.
  • Extensive use datastores like RDBMS, key-value stores, etc.
  • Experience with Development & Debugging with dev environments like Visual Studio or Visual Studio Code
  • HTML, CSS, JavaScript, ASP.NET, REST, jQuery
  • Familiarity with browser automation tools like Selenium, Puppeteer or Playwright.
  • Familiarity with mobile automation tools like Appium
  • Familiarity with LLMs and AI ChatBots
  • Prompt Engineering, Prompt Writing/Tuning, Azure DevOps, GIT
  • Azure Open AI, Azure Foundry
  • Experience in leading technical projects and supporting architectural decisions with data.

What the JD emphasized

  • evaluation platform
  • data sampling
  • AI data collection
  • data analysis
  • insight generation
  • prompt engineering
  • Copilot

Other signals

  • evaluation platform
  • AI data collection
  • data analysis
  • insight generation
  • prompt engineering
  • Copilot