Senior Machine Learning Engineer, Ads Experimentation & Measurements

Unity Unity · Enterprise · Mountain View, CA · AI & Machine Learning

Senior Machine Learning Engineer to lead the evolution of the Ads Experimentation Platform at Unity. This role focuses on validating and optimizing global advertising ecosystem by bridging advanced statistical methodology with large-scale engineering. Responsibilities include evaluating ad delivery systems, driving product decisions through analytics, advancing the experimentation platform with methodologies like CUPED and sequential testing, defining LTV proxy metrics, and driving automated ML experimentation pipelines. The role requires expertise in causal inference, experimental design, and large-scale data processing within the Ad Tech domain.

What you'd actually do

  1. Ad Delivery Expertise: Hands-on experience evaluating pacing and ad selection systems, with strong domain knowledge of the ads ecosystem (advertiser objectives, bidding/pricing dynamics, attribution, and marketplace mechanics) and practical familiarity with simulation-based measurement to pre-validate model and policy changes before live deployment.
  2. Product Analytics & Insights: Demonstrated ability to drive product decisions through analytics — defining metrics, building measurement frameworks, analyzing A/B test results, and translating experiment outcomes into clear, actionable insights for ML, Product, and Business partners
  3. Advance Experimentation Platform: design the statistical methodologies of our experimentation platform in a complex, budget-constrained environment. You will lead the implementation of variance reduction (CUPED), sequential testing, and interleaving frameworks to maximize sensitivity and accelerate evaluation cycles. You will also build the statistical foundations for automated pipelines that autonomously test and select optimal features and hyperparameters at scale.
  4. Define Long-Term Value (LTV) Proxy Metrics: Research and validate surrogate metrics that correlate highly with long-term user retention, churn, and value. You will provide ML teams with short-term signals that accurately predict long-term impact, enabling faster optimization for long-term growth.
  5. Drive Automated ML Experimentation: Build the statistical foundations for automated pipelines that autonomously test and select optimal features and hyperparameters. This reduces manual engineering overhead and accelerates the deployment of high-performing models at scale.
  6. Cross-Functional Technical Leadership: Serve as the lead subject matter expert on experimentation for ML, Product, and Engineering teams. You will ensure statistical rigor is integrated throughout the product lifecycle, from initial model training to live production auctions.

Skills

Required

  • 5+ years of experience in Data Science or Applied Research, specifically within Ad Tech, Marketplaces, or high-scale experimentation platforms.
  • MS or PhD in a quantitative field (Statistics, Economics, Computer Science, Operations Research, or equivalent).
  • Deep expertise in causal inference, experimental design, and frequentist/Bayesian statistics.
  • Strong programming skills in Python or Scala.
  • Experience with large-scale data processing frameworks like Spark, Snowflake, or BigQuery.
  • Practical experience implementing advanced testing methodologies like CUPED, interleaving, or switchback testing in production environments.
  • Ability to translate complex statistical concepts into clear product roadmaps.
  • Mentor engineering teams on experimental rigor.

Nice to have

  • Experience building or maintaining internal experimentation platforms at a major tech company.
  • Familiarity with the unique challenges of long-term value (LTV) prediction and surrogate metric design in mobile gaming or digital advertising.
  • Experience embracing AI as a strategic advantage in engineering, following established best practices for code quality and security

What the JD emphasized

  • critical to our growth
  • technical authority
  • bridge the gap
  • iterate faster and with higher confidence
  • high-velocity innovation
  • moving beyond standard A/B testing
  • next generation of causal inference and high-sensitivity evaluation methodologies
  • statistical methodologies
  • complex, budget-constrained environment
  • maximize sensitivity and accelerate evaluation cycles
  • automated pipelines that autonomously test and select optimal features and hyperparameters at scale
  • correlate highly with long-term user retention, churn, and value
  • accurately predict long-term impact
  • enabling faster optimization for long-term growth
  • reduces manual engineering overhead
  • accelerates the deployment of high-performing models at scale
  • lead subject matter expert
  • statistical rigor is integrated throughout the product lifecycle
  • high-volume, real-time data
  • applying these to high-volume, real-time data
  • mentor engineering teams on experimental rigor

Other signals

  • ML models
  • ads delivery pipelines
  • experimentation and evaluation roadmap
  • causal inference
  • high-sensitivity evaluation methodologies
  • pacing and ad selection systems
  • simulation-based measurement
  • A/B test results
  • variance reduction (CUPED)
  • sequential testing
  • interleaving frameworks
  • automated pipelines
  • LTV proxy metrics
  • surrogate metrics
  • automated ML experimentation
  • statistical rigor