Aiml - Data Scientist, Evaluation

Apple Apple · Big Tech · Seattle, WA +2 · Machine Learning and AI

This role focuses on developing and researching evaluation methods to improve the quality of Apple's user-facing products, particularly in search and AI features like Siri and Apple Intelligence. The Data Scientist will work with large datasets, apply advanced analytical methods including prompt engineering and building LLMs as judges, and collaborate with engineering teams to translate methodological developments into usable technologies. The goal is to guide product development and decisions through rigorous evaluation and data analysis.

What you'd actually do

  1. Research and develop evaluation methods to improve the quality of Apple user facing products, such as Siri and Apple Intelligence.
  2. Work with evaluation/experimentation engineering teams to get your methodological developments translated into technologies that product engineering will use every day.
  3. Work with large, complex data sets. Solve difficult, non-routine analysis problems, applying advanced analytical methods as needed, including prompt engineering and building LLMs as judges.
  4. Conduct analysis that includes data collection and quality control, requirements specification, processing and presentations.
  5. Build and prototype analysis pipelines iteratively to provide insights at scale.

Skills

Required

  • data science
  • machine learning
  • analytics
  • statistical data analysis
  • experiment design
  • experiment analysis
  • experiment interpretation
  • data quality assessment
  • risk assessment
  • SQL
  • Spark
  • Python
  • R
  • Scala
  • prompt engineering
  • building LLMs as judges

Nice to have

  • collaboration
  • project management
  • accuracy and variability in human annotation data
  • learning new technology

What the JD emphasized

  • 5 years of relevant work experience
  • Advanced degree in a quantitative field such as Statistics, Operational Research, Bioinformatics, Economics, Psychology, Computer Science, Sociology, Mathematics, Physics, or similar quantitative field.
  • Proficiency in data science, machine learning, and analytics, including statistical data analysis.
  • Experience crafting, conducting, analyzing, and interpreting experiments and investigations, especially on data quality, evaluation and risk assessment.
  • Strong programming skills, including data-querying skills (SQL and/or Spark, etc.) and experience with a scripting language for data processing and development (e.g., Python, R, or Scala).
  • Strong communication skills and the ability to naturally explain difficult technical topics (especially causal topics) to everyone from data scientists to engineers to business partners.

Other signals

  • driving product impact via measurement and evaluation
  • guide product development, decisions and directions through principled evaluation and rigorous use of data
  • improve search quality and guide feature development with data
  • Research and develop evaluation methods to improve the quality of Apple user facing products, such as Siri and Apple Intelligence
  • building LLMs as judges