Monitoring, tracing, and debugging LLM applications in production, including prompt and response logging, latency and cost tracking, and live quality metrics. Primary AI lifecycle stage: evaluation and application.
3,121 active AI roles across 225 companies in our index reference LLM observability as of today.
The companies with the most active LLM observability listings are: Amazon (304 roles), JPMorgan Chase (175 roles), Google (151 roles), Capital One (120 roles), Apple (92 roles).
LLM observability primarily belongs to the evaluation and application stages of the AI lifecycle. In current hiring, LLM observability roles concentrate at: agents (70%), serving infrastructure (9%).
The sectors with the most active LLM observability hiring are: Enterprise, Big Tech, Banking.
Monitoring, tracing, and debugging LLM applications in production, including prompt and response logging, latency and cost tracking, and live quality metrics.
Primary AI lifecycle stage: evaluation and application.
As of today, 3,121 active AI roles across 225 companies in our index reference LLM observability. Hiring concentrates at the agents (70%) and serving infrastructure (9%) stages. Most common sectors: Enterprise, Big Tech, Banking.
267 AI roles tagged llm_observability.
| Company | Title | Sector | AI score | Other tags |
|---|---|---|---|---|
| Airbnb | Principal Engineer -In Bayesian, Large Foundational Systems, and Distributional Reinforcement Learning | Consumer | 9 | Agent orchestration · Multi-agent · Multimodal · Recommender systems · Frontier research |
| Spotify | Machine Learning Engineer - Personalization, Horizon | Consumer | 9 | Agent orchestration · Fine-tuning · Model serving · Recommender systems |
| Spotify | Senior Machine Learning Engineer - Personalization, Horizon | Consumer | 9 | Agent orchestration · Fine-tuning · Recommender systems · Multimodal |
| Airbnb | Senior Staff Machine Learning Engineer, Post Training | Consumer | 9 | Fine-tuning · Model serving · Inference infra · Guardrails · Multimodal |
| Zillow | Senior Machine Learning Engineer | Consumer | 9 | Agent orchestration · Multimodal · Evals · Guardrails · Model serving |
| Master's Fall Machine Learning Internship (ATG - Visual Search) | Consumer | 9 | Agent orchestration · Tool use · Model serving · Inference infra · Multimodal | |
| Principal Engineer, Agentic Engineering | Consumer | 9 | Agent orchestration · Agent research · Evals · Guardrails · Tool use | |
| Sr. Data Scientist, Responsible AI | Consumer | 9 | Evals · Guardrails · Agent research · Multimodal | |
| Zillow | Principal Machine Learning Engineer, Agentic AI | Consumer | 9 | Agent orchestration · Multimodal · Evals · Guardrails · Model serving · Agent research |
| Zillow | Senior Applied Scientist, Agentic AI | Consumer | 9 | Agent orchestration · Tool use · Evals · Fine-tuning · Agent research |
| Roblox | Senior Machine Learning Engineering Manager | Consumer | 9 | Multimodal · Vision · Model serving · Fine-tuning |
| Uber | Sr Staff Agentic Systems Engineer | Consumer | 9 | Agent orchestration · Agent research · Tool use · Model serving |
| Uber | Senior Staff Machine Learning Engineer – Moonshot AI | Consumer | 9 | Multimodal · Vision · Audio & speech · Evals · Fine-tuning · RAG · Model serving · Recommender systems |
| Staff Research Engineer, Post-training & Evaluation | Consumer | 9 | Fine-tuning · Evals · Frontier research · RL post-training | |
| Uber | Senior Applied Scientist – AI Red Teaming & Model Risk | Consumer | 9 | Evals · Guardrails · Agent orchestration · Tool use · Agent research |
| Zillow | Distinguished Scientist | Consumer | 9 | Agent orchestration · Agent research · Multi-agent · Fine-tuning · RL post-training · Evals · Multimodal |
| Uber | Staff ML Engineer, Generative AI | Consumer | 9 | Agent orchestration · Tool use · Evals · Guardrails · RAG · Fine-tuning · Model serving · Multimodal · Audio & speech |
| Zillow | AI Applied Scientist - PhD Intern, Foundational AQ & EQ | Consumer | 9 | Agent orchestration · Fine-tuning · RL post-training |
| Zillow | AI Applied Scientist - PhD Intern, Next-Gen Agentic and Multi-Modal Home Exploration Experience | Consumer | 9 | Agent orchestration · Agent research · Multimodal · Vision · Tool use · Fine-tuning |
| Airbnb | Senior Staff Machine Learning Engineer, Data & Eval | Consumer | 9 | Evals · Guardrails · RAG · Agent orchestration · Tool use · Fine-tuning · Synthetic data |
| Instacart | Machine Learning Engineer, PhD Intern | Consumer | 9 | RAG · Fine-tuning · Inference infra · Model serving · Recommender systems · Search & ranking · Agent research · Evals |
| Senior Machine Learning Engineer, Ads Foundational Representations | Consumer | 8 | Fine-tuning · Multimodal · Recommender systems · Search & ranking · Inference infra · Model serving | |
| Zillow | Principal Product Technologist | Consumer | 8 | Agent orchestration · RAG · Tool use |
| DoorDash | Software Engineer, Machine Learning Infrastructure - Gen AI | Consumer | 8 | Agent orchestration · Tool use · Evals · Guardrails · RAG · Vector DB · Fine-tuning · Inference infra · Model serving |
| Chegg | Senior Software Engineer - Agentic AI Applications | Consumer | 8 | Agent orchestration · Tool use · Evals · Guardrails · RAG · Vector DB · Fine-tuning · Model serving · Multimodal |
| Roblox | Senior Data Scientist - Generative AI | Consumer | 8 | Evals · Fine-tuning · Agent orchestration · Multimodal · Code gen |
| Duolingo | Senior AI Engineering Manager | Consumer | 8 | Recommender systems · Fine-tuning · Model serving · Evals |
| Duolingo | Senior AI Engineering Manager | Consumer | 8 | Recommender systems · Fine-tuning · Model serving · Evals |
| Spotify | Senior Machine Learning Engineer - Content Intelligence | Consumer | 8 | Model serving · Inference infra · RAG · Vector DB · Fine-tuning · Agent orchestration · Multimodal · Evals |
| Oura | AI Fullstack Engineer, Health Intelligence | Consumer | 8 | Agent orchestration · Tool use · Evals · RAG · Fine-tuning · Model serving |