Staff Software Engineer, AI Search

Databricks Databricks · Data AI · Mountain View, CA · Engineering

Staff Software Engineer for AI Search at Databricks, focusing on building and scaling next-generation Search products including Vector Search, Keyword Search, and Search Quality. The role involves driving the architecture and evolution of a high-performance, cost-efficient Search stack to enable customers to build AI applications. Responsibilities include indexing, storage, query execution, ranking, and partnering with infrastructure, product, and research teams.

What you'd actually do

  1. Drive the architecture and evolution of the Search product, delivering scalable, low-latency, and high-quality retrieval across massive multimodal datasets.
  2. Identify and solve challenges in indexing, storage, query execution, and ranking — blending innovations in vector search, keyword search, and hybrid retrieval techniques.
  3. Partner with infrastructure, product, and research teams to shape APIs and developer experiences that make it easy to build production-grade, AI-powered and search-driven applications.
  4. Lead end-to-end engineering efforts — from design to production rollout — ensuring high standards for performance, correctness, and reliability.
  5. Make direct, high-impact contributions to core systems powering search, discovery, and recommendation experiences.

Skills

Required

  • building and operating large-scale distributed systems
  • Search technologies (vector, keyword, hybrid)
  • indexing
  • ranking
  • retrieval infrastructure
  • relevance optimization
  • storage systems
  • recommendation engines
  • database internals
  • algorithms
  • data structures
  • system design
  • leading architecture for performance-sensitive, large-scale systems
  • communication skills
  • collaboration

Nice to have

  • mentoring engineers
  • fostering technical excellence

What the JD emphasized

  • 10+ years of experience building and operating large-scale distributed systems.
  • Deep expertise in Search technologies (vector, keyword, hybrid) and related domains such as indexing, ranking, retrieval infrastructure, and relevance optimization.
  • Experience leading architecture for performance-sensitive, large-scale systems.

Other signals

  • building and scaling our next-generation Search product
  • enables customers to build AI applications with accuracy, speed, and scale
  • powering search, discovery, and recommendation experiences