Scale AI

Research Scientist role focused on designing and building evaluation measures, harnesses, and datasets for frontier AI systems, with a focus on identifying and mitigating risks. The role involves collaboration with external agencies and publishing findings, bridging AI research and policy.

Research Scientist focused on agent robustness, AI safety, and risk evaluations. The role involves researching AI agent capabilities, designing tests for harmful actions, creating exploits and mitigations for failure modes, and characterizing risks in multi-agent systems. Experience with post-training techniques like RLHF and published research in generative AI is required.

Research Scientist role focused on designing methods, systems, and experiments for AI controls and monitoring, ensuring advanced AI models and agents remain aligned with intended goals, even in high-stakes or adversarial environments. This includes developing monitoring techniques, researching layered control mechanisms, designing red-team simulations, and collaborating with policymakers.

Manager for a GenAI research team focused on evaluation, post-training, agents, and RL environments. The role involves leading a team, defining research roadmaps, driving execution, and collaborating cross-functionally. Requires a strong research background with publications and experience in fast-paced environments.

Scale AI is seeking a Staff Machine Learning Research Scientist to lead the development of novel evaluation methodologies, metrics, and benchmarks for large language models (LLMs). This role focuses on defining and measuring the capabilities and limitations of frontier LLMs, driving research that informs internal roadmaps and the broader community. Responsibilities include researching existing evaluation techniques, designing new benchmarks, implementing scalable evaluation pipelines, publishing findings, and mentoring junior researchers. The ideal candidate has 5+ years of experience in LLMs/NLP, a strong publication record, and experience leading research teams.

Scale AI is seeking a Staff Machine Learning Research Engineer focused on post-training algorithms for complex agents in enterprise GenAI applications. The role involves building a next-generation Agent RL training platform, integrating cutting-edge research, and training state-of-the-art models for enterprise customers, including cybersecurity and healthtech use cases. Experience with LLM training, post-training methods like RLHF/RLVR, and publications in top conferences are required.

Scale AI is seeking an ML Systems Research Engineer to work on building algorithms for their next-gen Agent RL training platform, supporting large-scale training, and researching/integrating state-of-the-art technologies to optimize ML systems. The role involves post-training state-of-the-art models for enterprise engagements and creating next-gen agent training algorithms for multi-agent/multi-tool rollouts.

Research Engineer focused on building and training advanced AI agents for enterprise GenAI applications, utilizing post-training and agent-building algorithms on real-world datasets to achieve state-of-the-art results.

This role focuses on researching and building synthetic data pipelines and agents to improve enterprise GenAI models. It involves creating agents for trace analysis, contributing to an agent-building framework, and training state-of-the-art models using post-training and agent-building algorithms.

Machine Learning Research Scientist focused on reasoning in LLMs, specifically for agentic systems like browser and software engineering agents. The role involves studying critical data types, identifying effective data sources and methodologies to improve LLM reasoning, and contributing to research while collaborating with engineering teams to implement solutions.

Research Scientist focused on LLM post-training techniques (SFT, RLHF, reward modeling) to enhance text and multimodal capabilities. Involves optimizing data curation, analyzing model behavior, and publishing findings.

Research Scientist role focused on building state-of-the-art AI agents, studying essential data types for agents like browser and SWE agents, and guiding data strategy to advance intelligent, adaptable AI agents. The role involves contributing to research publications, collaborating with customer researchers, and translating advancements into scalable solutions.

Scale AI is seeking a Tech Lead/Manager for their LLM Evals Research team. This role involves leading a team to develop and implement novel evaluation methodologies, metrics, and benchmarks for large language models, focusing on areas like instruction following, factuality, robustness, and fairness. The position requires research into LLM evaluation techniques, communication with clients and internal teams, implementation of scalable evaluation pipelines, and publishing research findings. The ideal candidate has extensive experience in LLMs, NLP, and Transformer modeling, with a proven track record of research impact and team leadership.

Internship role for university students to contribute to generative AI projects, focusing on training models for complex reasoning and identifying failure modes. Involves participation in focus groups and team-based projects, with flexible, remote work.