Principal Data Scientist - Safety

Roblox Roblox · Consumer · San Mateo, CA · Data Science

Roblox is seeking a Principal Data Scientist to focus on content safety and moderation. This role involves establishing and monitoring prevalence measurement methodologies, developing ground truth datasets for harmful content, conducting exploratory analysis on threat landscapes, designing and analyzing experiments for safety features, and leveraging causal inference to measure effectiveness. The scientist will collaborate with Product, Engineering, Legal, Policy, and Compliance teams, and partner with ML and Data Engineering teams to ensure statistically sound frameworks. The ideal candidate has extensive experience in data science, scripting languages, big data tools, and a background in safety or moderation systems for content platforms.

What you'd actually do

  1. Establish and monitor robust prevalence measurement methodologies to accurately quantify the overall level of harmful content and behavior on the platform, providing the authoritative source of truth for organizational safety goals.
  2. Develop and validate comprehensive ground truth datasets for harmful UGC and violative on-platform behaviors, ensuring high quality and alignment with the latest platform policies.
  3. Deepen our understanding of violations by conducting exploratory analysis on current and emerging threat landscapes, providing data-backed recommendations on where Product and Engineering should strategically invest detection and moderation resources.
  4. Design, implement, and analyze sophisticated experiments for new moderation and safety enforcement features, communicating results to our primary partners in Product and Engineering to guide development, while also collaborating closely with Policy, Compliance, and Legal teams for full strategic alignment.
  5. Leverage advanced causal inference methodologies to accurately measure the effectiveness and potential systemic impacts of various safety initiatives on player experience and platform integrity.

Skills

Required

  • 10+ years of industry experience in data science, economics, analytics, or machine learning engineering
  • 7+ years of experience using scripting languages (Python, R), and big data query/processing languages and tools such as SQL, Hive, Spark, and Airflow
  • Ability to apply creative first-principles reasoning to solve ambiguous problems
  • Experience developing large-scale safety or moderation systems as well as experience with content platforms, specifically user-generated content
  • Advanced Degree and/or PhD in Statistics, Computer Science, Physics, Applied Math, Economics, or other related quantitative fields

Nice to have

  • Knowledge of ML and Deep Learning either via formal training or industry experience

What the JD emphasized

  • Establish and monitor robust prevalence measurement methodologies
  • Develop and validate comprehensive ground truth datasets
  • sophisticated experiments for new moderation and safety enforcement features
  • advanced causal inference methodologies
  • statistically sound ground truth and measurement frameworks

Other signals

  • measurement methodologies
  • ground truth datasets
  • detection and mitigation of harmful user-generated content
  • safety threats
  • detection systems