Applied Scientist Ii, Alexa International Team

Amazon Amazon · Big Tech · Bellevue, WA · Applied Science

Applied Scientist II on the Alexa International Team at Amazon, focusing on developing novel algorithms and modeling techniques for Large Language Models (LLMs) and multimodal systems. The role involves fine-tuning/post-training LLMs, building evaluation metrics, and contributing to end-to-end delivery from research to production, impacting international customers with digital assistant technology.

What you'd actually do

  1. Build novel online & offline evaluation metrics and methodologies for multimodal personal digital assistants.
  2. Fine-tune/post-train LLMs using techniques like SFT, DPO, RLHF, and RLAIF.
  3. Set up experimentation frameworks for agile model analysis and A/B testing.
  4. Collaborate with partner teams on LLM evaluation frameworks and post-training methodologies.
  5. Contribute to end-to-end delivery of solutions from research to production, including reusable science components.

Skills

Required

  • PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
  • Experience in patents or publications at top-tier peer-reviewed conferences or journals
  • Experience programming in Java, C++, Python or related language
  • Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing

Nice to have

  • Experience in professional software development
  • PhD

What the JD emphasized

  • novel algorithms and modeling techniques
  • LLM evaluation & tooling
  • pushing boundaries
  • swiftly delivering impactful solutions
  • iterating based on user feedback
  • collaborate effectively with cross-functional teams
  • novel online & offline evaluation metrics and methodologies
  • fine-tune/post-train LLMs
  • LLM evaluation frameworks and post-training methodologies
  • end-to-end delivery of solutions from research to production

Other signals

  • LLMs
  • multimodal systems
  • deep learning
  • generative models
  • international products and services
  • text, voice, and vision domains
  • LLM evaluation & tooling
  • pushing boundaries
  • fast-paced environments
  • complex challenges
  • swiftly delivering impactful solutions
  • iterating based on user feedback
  • collaborate effectively with cross-functional teams