Staff Backend Software Engineer

Databricks Databricks · Data AI · New York, NY · Engineering

Staff Backend Software Engineer on the AI Platform team at Databricks, responsible for building and improving LLM infrastructure, including model serving, agent support, and Vector Search, to power customer AI workloads.

What you'd actually do

  1. Build LLM infrastructure powering trillions of tokens per week for customers through partner models (OpenAI, Anthropic, Gemini) and self hosted models (Qwen, GPT-OSS, Llama)
  2. Improve reliability, latency, and efficiency of distributed AI workloads
  3. Collaborate with platform, infra, and ML teams to deliver seamless end-to-end experiences
  4. Shape how developers and data scientists build and interact with AI on Databricks

Skills

Required

  • 8+ years of experience in backend or infrastructure engineering
  • Scala, Go, or Python
  • distributed systems
  • scalable APIs
  • cloud-native infrastructure
  • service-oriented architecture
  • deployment pipelines
  • system observability
  • product and ownership mindset

Nice to have

  • real-time serving
  • ML infrastructure
  • GPU orchestration
  • SageMaker, Vertex AI, or Azure ML
  • MLflow, PyTorch, Ray, vLLM, SGLang
  • developer platforms
  • internal tools supporting AI workflows

What the JD emphasized

  • high-agency, high-visibility team
  • frontier of AI infrastructure
  • deep ties to research, product, and real-world enterprise use cases
  • Mosaic AI is one of our fastest-growing businesses
  • building the infrastructure that powers the next generation of AI

Other signals

  • LLM infrastructure
  • distributed AI workloads
  • AI agents
  • model serving
  • Vector Search