Senior/staff Applied Scientist

Glean Glean · Enterprise · Mountain View, CA · Data

Senior/Staff Applied Scientist at Glean, an enterprise Work AI platform. The role focuses on developing and maintaining an A/B experimentation platform, conducting end-to-end evaluations of AI use cases (including summarization, reasoning, tool use, coding), and designing best practices for evaluation problems. The role requires strong statistics/ML skills, Python proficiency, SQL, and experience in B2B SaaS and A/B testing platforms.

What you'd actually do

  1. Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
  2. Develop and maintain our A/B experimentation platform based on stakeholder feedback
  3. Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams
  4. Conduct end to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results.
  5. Break down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design.

Skills

Required

  • Statistics
  • Machine Learning
  • Python
  • SQL
  • dbt
  • A/B experimentation
  • product KPIs
  • dashboarding
  • data analysis

Nice to have

  • B2B SaaS experience
  • ranking
  • LLM
  • AI agents

What the JD emphasized

  • craft and care required for enterprise trust
  • end to end evaluation
  • evaluation set generation
  • evaluation criteria
  • evaluation problems

Other signals

  • AI agents
  • LLM
  • evaluation
  • experimentation platform
  • enterprise trust