Senior Software Engineer, Backend Platform

Harvey Harvey · AI Frontier · San Francisco, CA · Engineering

This role is for a Senior Software Engineer focused on building and operating the backend platform for an AI company. The engineer will design and implement shared frameworks and libraries to abstract common concerns, improve developer experience, and enhance reliability and performance across all services. The goal is to enable product teams to ship features faster and more safely by providing a strong foundational platform. Collaboration with the Model Infrastructure team on GenAI-specific challenges like high-throughput inference and streaming interactions is also a key aspect.

What you'd actually do

  1. Develop and maintain Harvey’s internal backend frameworks and libraries that provide common capabilities (API routing, service lifecycle management, caching and messaging primitives, error handling interfaces, etc.), so product teams don’t have to reinvent them
  2. Create and improve APIs, service templates, and versioned interfaces that establish consistent patterns for building new services and features
  3. Introduce and champion modern backend architecture patterns like asynchronous I/O (asyncio) and streaming data processing, continually evolving our platform for better performance and scalability
  4. Design Harvey-specific abstractions and domain-specific frameworks—covering cross-cutting concerns (e.g., authorization, streaming) and areas like data governance and event processing—to provide product engineers with these capabilities out of the box.
  5. Embed reliability and observability into the platform by building in tracing, metrics, and automated tests (shift-left), ensuring services built on our foundation are robust and easy to monitor

Skills

Required

  • 5+ years of software engineering experience
  • building scalable backend systems or internal developer platforms
  • Proficiency in Python (or similar languages)
  • deep knowledge of backend development fundamentals (APIs, data stores, concurrency, distributed systems)
  • Hands-on experience with web frameworks and service architectures (e.g. Flask/FastAPI, Bounded context services, microservices)
  • understanding of designing clean, versioned APIs
  • Familiarity with caching, messaging, and database technologies (Redis, Kafka, SQL/NoSQL databases, Vector databases, etc.)
  • how to use them effectively for high performance and reliability
  • A track record of writing high-quality, well-tested code
  • using tools (unit/integration testing, static typing, CI) to catch issues early and ensure reliability
  • Strong problem-solving skills
  • a passion for improving developer experience
  • Excellent collaboration and communication skills
  • ability to work across teams and incorporate feedback

Nice to have

  • Experience contributing to or maintaining internal libraries/frameworks

What the JD emphasized

  • cohesive backend platform
  • amplify the productivity and quality
  • shipping reliable, observable, and performant experiences quickly and safely
  • high-throughput model inference
  • streaming and long-running API interactions