Currently tracking 25 active AI roles, up 183% versus the prior 4 weeks. Primary focus: Agent · Engineering. Salary range $115k–$451k (avg $239k).
| Title | Stage | AI score |
|---|---|---|
| Agent Evaluation Engineer This role focuses on building and managing evaluation pipelines, metrics, and automated systems to test the behavior, accuracy, and reliability of AI agents before release. It involves defining benchmarks, curating datasets, integrating evaluation into CI/CD, and monitoring agents in production. | Eval GateAgent | 8 |
| Development Engineer in Test (SDET) – ML & LLM Systems This role focuses on evaluating, validating, and measuring LLM behavior within NLP pipelines and ML quality frameworks. The engineer will design and implement automated test strategies and frameworks for ML models, NLP systems, and backend services, including model validation, benchmarking, and drift detection. Experience with LLM evaluation frameworks and testing ML models is required. | Eval Gate |
| 7 |
| Software Development Engineer in Test (SDET) – ML & LLM Systems Software Development Engineer in Test (SDET) focused on ML & LLM Systems, specifically evaluating and validating LLM behavior, performance, and reliability. The role involves designing and implementing automated test strategies, frameworks, and pipelines for ML models, NLP systems, and LLM evaluations, ensuring quality before deployment. | Eval GatePost-train | 7 |
| Agentic AI Test Engineer Seeking an AI Agentic Test Engineer to build automated evaluation frameworks using LLM-as-a-Judge patterns and maintain web/API test suites. Focus on agent evaluation and full-stack automation in Python, with experience in CI/CD and troubleshooting. | Eval GateAgent | 7 |