Staff Software Engineer, Agents

Harvey Harvey · AI Frontier · New York, NY · Engineering

Staff Software Engineer, Agents role at Harvey, focusing on building and optimizing AI agents for legal professionals. Responsibilities include designing agent environments, managing context windows, developing tools, creating evaluations, and optimizing performance through prompt engineering and model selection. The role involves working with LLM APIs, agent frameworks, and shipping user-facing products.

What you'd actually do

  1. Partner with customers and PMs to understand legal workflows, design practical evaluations that capture what “excellent” means, and ship agents that get the job done.
  2. Optimize agent performance through prompt engineering, model selection, tool design, skill writing, context window management, and eval harness development.
  3. Work with our model infra team to design and implement infrastructure for low-latency agent execution, including caching strategies, parallel tool calls, or subagent patterns.
  4. Improve our observability and instrumentation to profile agent behavior, identify bottlenecks, and drive optimization decisions.
  5. Stay current on new developments in agentic systems and bring those learnings back to the products we build.

Skills

Required

  • Python
  • LLM APIs
  • agent frameworks
  • shipping user-facing products
  • software engineering experience

Nice to have

  • prompt engineering
  • model selection
  • tool design
  • skill writing
  • context window management
  • eval harness development
  • low-latency agent execution
  • caching strategies
  • parallel tool calls
  • subagent patterns
  • observability
  • instrumentation

What the JD emphasized

  • shipping impactful products
  • practical evaluations to drive task completion quality
  • shipping user-facing products

Other signals

  • building agentic AI systems
  • enterprise-grade platform
  • customer-facing products