Annotation Qa Analyst - Content Platform

Spotify Spotify · Consumer · New York, NY · Experience

The Annotation QA Analyst will create labeled datasets for training and evaluating ML/AI models within Spotify's Content Platform. This role involves working with annotation, data quality, and QA processes in ML/AI environments, including LLM-driven workflows and human-in-the-loop systems. The analyst will review large-scale datasets across various modalities and understand the machine learning lifecycle and emerging AI tools.

What you'd actually do

  1. create a broad collection of labeled datasets that Content Platform teams use to train, evaluate, and better understand models and systems.

Skills

Required

  • experience working with annotation, data quality, or QA processes in ML/AI environments
  • familiarity with LLM or AI-driven annotation workflows and human-in-the-loop systems
  • comfortable reviewing large-scale datasets across different modalities such as text, audio, images, or video
  • care about quality and consistency, and bring a structured approach to evaluating data
  • communicate clearly and can explain complex ideas in a simple, accessible way
  • collaborate well with cross-functional partners in fast-moving environments
  • solid understanding of the machine learning lifecycle, from data collection to deployment
  • comfortable working with emerging AI tools and agent workflows

Nice to have

  • familiarity with SQL or music metadata standards

What the JD emphasized

  • annotation, data quality, or QA processes in ML/AI environments
  • LLM or AI-driven annotation workflows and human-in-the-loop systems
  • reviewing large-scale datasets
  • solid understanding of the machine learning lifecycle
  • working with emerging AI tools and agent workflows

Other signals

  • labeled datasets
  • train, evaluate, and better understand models
  • ML and AI-driven development
  • annotation, data quality, or QA processes in ML/AI environments
  • LLM or AI-driven annotation workflows
  • human-in-the-loop systems
  • reviewing large-scale datasets
  • machine learning lifecycle
  • emerging AI tools and agent workflows