Senior Domain Expert Lead- Stem (contract), Agi - Data Services

Amazon Amazon · Big Tech · Boston, MA · Research Science

This role focuses on providing domain expertise to human-in-the-loop and model-in-the-loop data pipelines for LLM training and evaluations. The Senior Domain Expert Lead will ensure data quality, optimize data collection, and mentor junior team members, while staying updated on GenAI applications in their domain.

What you'd actually do

  1. Serve as a trusted domain advisor to cross-functional teams, providing strategic direction and specialized problem-solving support
  2. Champion domain knowledge sharing across multiple channels and teams to maintain data quality excellence and standardization
  3. Drive collaborative efforts with science teams to optimize output of complex data collections in your domain expertise, ensuring data excellence through iterative feedback loops
  4. Foster team excellence through mentorship and motivation of peers and junior team members
  5. Collaborate with AI teams to innovate model-in-the-loop and human-in-the-loop approaches, to ensure the collection of high-quality data, safeguarding data privacy and security for LLM training, and more.

Skills

Required

  • 2+ years of data scientist experience
  • 3+ years of data querying languages (e.g. SQL), scripting languages (e.g. Python) or statistical/mathematical software (e.g. R, SAS, Matlab, etc.) experience
  • 3+ years of machine learning/statistical modeling data analysis tools and techniques, and parameters that affect their performance experience
  • 1+ years of guiding and coaching a group of researchers experience
  • 1+ years of working with or evaluating AI systems experience
  • Master's degree in Science, Technology, Engineering, or Mathematics (STEM), or experience working in Science, Technology, Engineering, or Mathematics (STEM)
  • Experience applying theoretical models in an applied environment

Nice to have

  • Ph.D. in Science, Technology, Engineering, or Mathematics (STEM)
  • Knowledge of machine learning concepts and their application to reasoning and problem-solving
  • Experience in Python, Perl, or another scripting language
  • Experience in a ML or data scientist role with a large technology company
  • Experience in defining and creating benchmarks for assessing GenAI model performance
  • Experience working on multi-team, cross-disciplinary projects
  • Experience applying quantitative analysis to solve business problems and making data-driven business decisions
  • Experience effectively communicating complex concepts through written and verbal communication

What the JD emphasized

  • model-in-the-loop
  • human-in-the-loop
  • data quality
  • LLM training
  • evaluations
  • GenAI

Other signals

  • human-in-the-loop
  • model-in-the-loop
  • data pipelines
  • LLM training
  • evaluations