Member of Technical Staff (product Data Scientist, Search Quality)

Perplexity Perplexity · AI Frontier · Belgrade, Serbia · Search

Product Data Scientist focused on search quality, developing data-driven insights, designing and analyzing A/B tests, and creating pipelines for ranking quality, including data labeling with human and LLM feedback.

What you'd actually do

  1. Develop data-driven insights from user behavior to inform our product roadmap and accelerate adoption
  2. Formulate hypotheses and validate them by designing, running, and analyzing A/B tests
  3. Determine appropriate metrics and visualizations for tracking, and implement them in dashboards
  4. Design new pipelines that will help to deliver better ranking quality. From discovering new signals, producing metrics and construct data labeling pipelines with human and LLM feedback

Skills

Required

  • 4+ years of experience working as a data analyst or in a related role
  • Experience working on search-related products
  • Strong Python skills
  • Proficiency with SQL
  • Experience with Business Intelligence (BI) tools
  • Deep knowledge of statistics

Nice to have

  • Apache Spark
  • Databricks
  • development of LLM-as-a-judge systems

What the JD emphasized

  • designing online metrics and analyzing A/B experiments
  • construct data labeling pipelines with human and LLM feedback

Other signals

  • Develop data-driven insights from user behavior to inform our product roadmap and accelerate adoption
  • Formulate hypotheses and validate them by designing, running, and analyzing A/B tests
  • Determine appropriate metrics and visualizations for tracking, and implement them in dashboards
  • Design new pipelines that will help to deliver better ranking quality. From discovering new signals, producing metrics and construct data labeling pipelines with human and LLM feedback