Staff Software Engineer, Backend Platform

Harvey Harvey · AI Frontier · San Francisco, CA · Engineering

Staff Software Engineer, Backend Platform at Harvey, focusing on building and operating the core backend platform for GenAI-native applications. This role involves designing and implementing shared frameworks, libraries, and APIs to abstract common concerns, improve developer experience, and enhance reliability and performance. The engineer will collaborate with the Model Infrastructure team on challenges related to high-throughput model inference, streaming interactions, and retrieval/context handling, aiming to amplify engineering productivity and quality across the company.

What you'd actually do

  1. Develop and maintain Harvey’s internal backend frameworks and libraries that provide common capabilities (API routing, service lifecycle management, caching and messaging primitives, error handling interfaces, etc.), so product teams don’t have to reinvent them
  2. Create and improve APIs, service templates, and versioned interfaces that establish consistent patterns for building new services and features
  3. Introduce and champion modern backend architecture patterns like asynchronous I/O (asyncio) and streaming data processing, continually evolving our platform for better performance and scalability
  4. Design Harvey-specific abstractions and domain-specific frameworks—covering cross-cutting concerns (e.g., authorization, streaming) and areas like data governance and event processing—to provide product engineers with these capabilities out of the box.
  5. Embed reliability and observability into the platform by building in tracing, metrics, and automated tests (shift-left), ensuring services built on our foundation are robust and easy to monitor

Skills

Required

  • Python (or similar languages)
  • backend development fundamentals (APIs, data stores, concurrency, distributed systems)
  • web frameworks and service architectures (e.g. Flask/FastAPI, Bounded context services, microservices)
  • designing clean, versioned APIs
  • caching, messaging, and database technologies (Redis, Kafka, SQL/NoSQL databases, Vector databases, etc.)
  • writing high-quality, well-tested code
  • unit/integration testing
  • static typing
  • CI
  • problem-solving skills
  • improving developer experience
  • collaboration and communication skills

Nice to have

  • Experience contributing to or maintaining internal libraries/frameworks

What the JD emphasized

  • cohesive backend platform
  • amplify the productivity and quality
  • challenges unique to GenAI-native applications
  • high-throughput model inference
  • managing streaming and long-running API interactions
  • designing abstractions for retrieval, context handling, and prompt lifecycle

Other signals

  • building and operating the cohesive backend platform
  • design and implement shared frameworks and libraries
  • amplify the productivity and quality of engineering across the entire company
  • collaborate with Model Infrastructure team to tackle challenges unique to GenAI-native applications