Senior Data Scientist, Trust & Safety and Content Quality

Pinterest Pinterest · Consumer · Toronto, ON · Core Engineering

Senior Data Scientist role focused on Trust & Safety and Content Quality at Pinterest. The role involves designing and developing ML-assisted sampling techniques, applying statistical methods to measure the prevalence of unsafe content, and building large-scale data pipelines. It also includes orchestrating monitoring workflows, translating safety policies into LLM prompts, defining content quality standards, analyzing content distribution, and evaluating external links. The role requires extensive experience in data analysis, statistical modeling, and machine learning, with a strong interest in platform safety and measurement challenges.

What you'd actually do

  1. Design and develop ML-assisted sampling techniques, applying expertise in statistical methods to accurately measure the prevalence of unsafe content, treating complex multi-component interactions as distinct measurement units.
  2. Apply rigorous statistical methods, drawing on knowledge of all kinds of sampling methods and their proper statistical application for complicated use cases, to calculate prevalence rates for specific Trust & Safety policy violations (e.g., Adult content, Self-harm, Harassment, Misinformation) and to further expand and improve prevalence measurement.
  3. Build large-scale data pipelines to aggregate Pinner-generated queries, system responses, and recommended Pin images into a unified format for human and ML-based safety labeling.
  4. Partner cross-functionally to orchestrate “Offline” dashboards and robust “Online” production workflows for continuous safety monitoring.
  5. Collaborate closely with Trust & Safety teams to translate written safety policies into unified LLM prompts, coordinate BPO labeling queues, and calibrate labeler decision quality.

Skills

Required

  • 5+ years of experience analyzing data in a fast-paced, data-driven environment with proven ability to apply scientific methods to solve real-world problems on web-scale data.
  • Extensive experience solving analytical problems using quantitative approaches in Machine Learning, Statistical Modeling, Forecasting, Econometrics, or related fields, with a proven record of researching and implementing advanced methods on real-world measurement problems.
  • Strong interest and hands-on experience in platform safety, prevalence measurement, content quality measurement, adversarial testing, responsible data measurement, or Trust & Safety.
  • Deep familiarity with the measurement challenges of a complex ecosystem, including statistical interpretation of data across multimodal and unstructured content types.
  • Experience designing and calibrating measurement frameworks, managing complex logging tables (e.g., user/interaction/component data), and defining directional success metrics.
  • Experience using machine learning and deep learning frameworks, such as PyTorch, TensorFlow, or scikit-learn.
  • Strong quantitative programming (Python) and data manipulation skills (SQL/Spark); experience with complex ML pipelines and up-sampling.
  • A scientifically rigorous approach to analysis and data, with a well-tuned sense of skepticism, high intellectual curiosity

What the JD emphasized

  • ML-assisted sampling techniques
  • statistical methods to accurately measure prevalence of unsafe content
  • large-scale data pipelines
  • continuous safety monitoring
  • unified LLM prompts
  • content quality at a platform level
  • end-to-end content distribution funnel
  • quality of links to external websites

Other signals

  • ML-assisted sampling techniques
  • statistical methods to accurately measure prevalence of unsafe content
  • build large-scale data pipelines
  • orchestrate dashboards and production workflows for continuous safety monitoring
  • translate written safety policies into unified LLM prompts
  • define and evangelize what constitutes high-quality content
  • analyze and model the end-to-end content distribution funnel
  • evaluate the quality of links to external websites