Principal Software Engineer, Experimentation

Roblox Roblox · Consumer · San Mateo, CA · Software Engineering

Roblox is seeking a Principal Software Engineer to lead the evolution and scaling of their experimentation platform. This role involves designing and building scalable systems for online serving and offline analysis, supporting hundreds of experiments daily for millions of users. The engineer will focus on scaling the platform, simplifying experiment creation and analysis, and designing serving models for novel experimentation strategies, potentially leveraging AI. The role reports into the Foundation AI Organization but the core function is platform engineering for experimentation.

What you'd actually do

  1. Help scale our online experiment serving flow to scale 10x.
  2. Design and simplify the experiment creation and analysis flows for the next generation of internal and external customers leveraging AI when necessary.
  3. Design a scalable serving model to support novel experimentation strategies (like crossovers, multi-armed bandits) and support our Data Science partners.
  4. Partner with our Data Engineering and Data Platform teams to build our experimentation data processing workflows.
  5. Help build a reliable serving flow and a scalable data to support experimental analysis, gradual feature rollouts and human-centric reporting.

Skills

Required

  • Backend engineering experience with 10+ years of work experience
  • Solid programming skills in compiled languages such as C#, Go, C++, or others
  • knowledge of SQL
  • Designed and built backend platform systems that can handle over 1M QPS
  • Experience with big-data processing technologies such as Spark, Apache Druid, Flink etc.
  • data-driven focus for quality metrics and monitoring
  • Knowledge with REST, API design patterns, and scalable containerized systems and micro-services
  • B.Sc. equivalent in CS or sufficient experience

Nice to have

  • Experience in building experimentation platforms serving millions of users in preferably multi-sided marketplaces
  • Experience building and tuning Spark workflows at TB+ scale
  • Experience leveraging AI tooling to build simplified, human-centric experiences for internal customers

What the JD emphasized

  • over ~98 million daily users
  • platform already serves millions in QPS
  • scale 10x
  • over 1M QPS
  • leveraging AI when necessary
  • novel experimentation strategies
  • scalable data to support experimental analysis
  • Foundation AI Organization