Senior Data Engineer

Microsoft Microsoft · Big Tech · United States · Data Engineering

Senior Data Engineer for the Product Engineering Partnership pillar, focusing on telemetry, instrumentation, semantic modeling, experimentation, feedback pipelines, catalog management, external reporting, and analytics for Global Skilling products. This role involves technical leadership, owning telemetry contracts, building the experimentation data backbone, and ensuring insights flow back to product teams.

What you'd actually do

  1. Own telemetry contracts and instrumentation that ship with Global Skilling products (e.g., Microsoft Learn, ASN, Skilling Catalog)
  2. Design and operate experimentation data infrastructure, including A/B testing pipelines, metric definitions, and significance tooling
  3. Build catalog and content management data integrations across the Global Skilling product surface
  4. Lead semantic model design for product analytics in partnership with Reporting and Analytics teams
  5. Build and maintain external (end-customer) reporting pipelines for ASN and partner-facing products

Skills

Required

  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field
  • 3+ years of experience in business analytics, data science, software development, data modeling, or data engineering

Nice to have

  • 5+ years of experience building production data systems on Azure (ADLS, Synapse, Fabric, or Databricks) or comparable cloud platforms
  • Experience instrumenting product telemetry and owning end-to-end data contracts with product engineering teams
  • Experience with experimentation platforms, A/B testing data, or feature-flag analytics infrastructure
  • Distributed systems fundamentals
  • PySpark
  • SQL
  • Pipeline orchestration
  • Mentoring mid-level engineers
  • Master’s degree in Computer Science, Engineering, or related technical field
  • Experience with Microsoft internal data platforms (Cosmos, Scope, Conform, Aria)
  • Experience with catalog and content management data systems (e.g., Microsoft Learn, ASN, credential catalogs)
  • Experience building external or end-customer reporting (e.g., ASN partner reporting)
  • Experience designing semantic models for product analytics and decision support
  • Experience with AI-driven telemetry discovery (e.g., agent-assisted instrumentation, anomaly detection, intelligent feedback loops)

What the JD emphasized

  • Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years of experience in business analytics, data science, software development, data modeling, or data engineering
  • 5+ years of experience building production data systems on Azure (ADLS, Synapse, Fabric, or Databricks) or comparable cloud platforms
  • Demonstrated experience instrumenting product telemetry and owning end-to-end data contracts with product engineering teams
  • Experience with experimentation platforms, A/B testing data, or feature-flag analytics infrastructure
  • Demonstrated distributed systems fundamentals; experience with PySpark, SQL, and pipeline orchestration