AI Engineer, Model Quality and Performance

Cerebras Cerebras · Semiconductors · Headquarters +1 · Software

AI Engineer focused on model quality and performance for Cerebras' inference offerings. The role involves designing and building AI-driven systems to measure model quality at scale, using AI agents to create custom eval suites, automate eval execution, and synthesize quality and performance data into a usable view.

What you'd actually do

  1. Design eval suites with AI agents in the loop.
  2. Build custom evals for target customers by orchestrating AI agents to mine trajectories from their workloads and synthesize representative eval sets.
  3. Automate eval execution end-to-end with AI-driven pipelines on top of standard tooling (Docker, Git, CI).
  4. Build automations to forecast and benchmark model performance on Cerebras for our top customers, including modeling how fast customer-specific workloads will run in production.
  5. Build product-quality tooling that synthesizes quality + performance data into a single, easy-to-use view.

Skills

Required

  • AI agents
  • Claude (or equivalent)
  • Docker
  • Git
  • automation stack
  • tooling design

Nice to have

  • Performance-tuning experience on custom silicon, GPUs, or FPGAs
  • evals for agentic / coding / long-context / multimodal use cases
  • open-source eval frameworks (EvalScope, lm-eval-harness, etc.)
  • AI agents

What the JD emphasized

  • Experience building AI agents.
  • You ship real systems with Claude (or equivalent) as a force multiplier.
  • You've built things that would have been infeasible solo without AI agents in the loop.
  • Experience designing evals for agentic / coding / long-context / multimodal use cases.

Other signals

  • AI agents for custom eval suites
  • Automate eval execution end-to-end
  • Build product-quality tooling for quality + performance data