The Data Scientist will focus on developing evaluation and benchmarking datasets for generative AI capabilities within the Amazon Quick Suite enterprise AI platform. This includes leveraging LLMs for synthetic data generation, creating ground truth datasets, leading human annotation initiatives, and contributing to Responsible AI efforts to ensure enterprise-readiness, safety, and effectiveness of AI at scale.

The Data Scientist II will focus on developing evaluation and benchmarking datasets for enterprise AI features, specifically for Amazon Quick Suite. This involves leveraging Generative AI techniques, LLMs for synthetic data generation, and LLM-as-a-judge settings to assess model performance, ensure data quality, and contribute to Responsible AI initiatives. The role also includes building scalable data pipelines and tools for continuous evaluation.

The Data Scientist will focus on developing evaluation and benchmarking datasets for generative AI capabilities within the Amazon Quick Suite enterprise AI platform. This includes leveraging LLMs for synthetic data generation, creating ground truth datasets, leading human annotation initiatives, and contributing to Responsible AI efforts to ensure enterprise-readiness, safety, and effectiveness of AI at scale.

The AI Principal PMT for Alexa Responsible AI will define the standard for how Alexa earns and keeps customer trust. This role owns the product discipline of Responsible AI, defining customer experiences for safety guardrails, trust signals, and evaluation frameworks. The PMT will set product vision and strategy, lead cross-functional alignment across Applied Science, Engineering, Legal, Policy, and UX, and ensure the full responsible product experience including safety, privacy, and security. The role requires technical depth in LLMs and AI safety, understanding how models fail and writing requirements for safety model development and evaluation system design. The PMT will also mentor other PMs and influence Responsible AI scaling across Alexa.

Applied Scientist role focused on automated reasoning, privacy, and sovereignty within AWS. The role involves solving complex problems, designing and implementing solutions, and providing cross-organizational technical influence. Requires a PhD or Master's with significant applied research experience in areas like SAT, SMT, theorem proving, symbolic simulation, program analysis, or type systems. Experience with specific programming languages like O'Caml, Dafny, Haskell, Lean, or Rust is preferred.

Senior Software Development Engineer role focused on applying Automated Reasoning to verify Generative AI outputs, specifically addressing hallucinations within AWS services. The role involves designing and building new services and capabilities at scale, contributing to the evolution of the Automated Reasoning Checks (ARc) service, and making automated reasoning more accessible within AWS.

Senior Applied Scientist role focused on developing evaluation frameworks and data collection protocols for robotic capabilities. The role involves designing how to measure, stress-test, and improve robot behavior, building infrastructure for teleoperation, evaluation, and learning, and analyzing results to identify performance gaps. It requires expertise in robotics, ML, and human-in-the-loop systems, with a focus on turning capability goals into measurable evaluation systems.

This role is for a Software Development Manager on the Agentic AI organization's Evaluations & Optimization team at AWS. The manager will lead a team of engineers to build systems for assessing the quality, performance, and reliability of GenAI and agentic systems, as well as optimization solutions. The work involves deep learning, distributed systems, and evaluation science, focusing on building infrastructure and tooling for evaluation workflows.

The Applied Scientist will develop and maintain LLM-as-a-Judge systems and auditing frameworks to ensure the quality of data used for training and evaluating Amazon Nova models, impacting LLM products and services.

Applied Science Manager for AGI team focusing on quality automation, auditing, and evaluation of LLMs and multimodal systems. Leads a team of scientists to develop quality strategies, auditing frameworks, and research new methodologies to ensure data integrity and model performance. Manages team development, cross-functional communication, and drives research into data impact and utility measurement for AI models.