Engineering Analyst, Kids and Learning Trust and Safety

Google Google · Big Tech · Dublin, Ireland

This role supports the launch of Generative AI search experiences and education efforts, focusing on responsible AI capabilities. The analyst will perform data analysis to identify and combat abuse, develop datasets and run evaluations for Gen AI products, establish metrics for AI issues, and improve model safety through data analysis. The role requires experience in data analysis, project management, and familiarity with ML model performance or LLMs.

What you'd actually do

  1. Perform complex data analyses to derive insights that help identify and fight abuse across our products. Use technical judgment to drive project delivery, challenge proposals and identify risks.
  2. Develop datasets and run evals for engineers to evaluate and improve Search and Education Gen AI products. Enhance our operational workflows via process improvements and identification of automation opportunities (including using AI to transform operations).
  3. Help establish metrics to benchmark Generative-AI issues in the Education space and quantify improvements. Provide end to end ownership of identifying gaps in protections, build consensus on path to mitigation, and land timely execution.
  4. Collaborate with other teams and functions (e.g., Engineering, PM, Policy, Responsible AI Testing, etc.). Perform on-call responsibilities on a rotating basis, including weekend coverage.
  5. Improve classifiers and base model level safety through data analysis.

Skills

Required

  • Bachelor's degree or equivalent practical experience.
  • 5 years of experience managing projects and defining project scope, goals, and deliverables.
  • 5 years of experience in data analysis, working with databases and querying (e.g., SQL, MySQL, MapReduce).

Nice to have

  • 5 years of experience or familiarity with one or more of the following languages: SQL, R, Python, or C++.
  • Experience working with abuse, spam, fraud or malware.
  • Experience working on product policy analysis and identifying policy risks.
  • Experience using statistical analysis and hypothesis testing.
  • Experience analyzing ML models performance or working on LLMs.
  • Excellent influencing and written and verbal communication skills, including ability to work in a dynamic environment.

What the JD emphasized

  • Responsible AI Testing

Other signals

  • Generative AI search experience
  • responsible learning capabilities
  • evals for engineers
  • Generative-AI issues
  • ML models performance
  • LLMs