Staff Software Engineer - Machine Learning (search)

Databricks Databricks · Data AI · Bangalore, India · Engineering - Pipeline

Staff Software Engineer focused on Machine Learning for Search at Databricks. The role involves developing and deploying ML-based search and discovery relevance models, enhancing search ranking, improving query understanding, building robust evaluations, and applying LLMs to search relevance. The goal is to enable seamless search at scale for Databricks customers.

What you'd actually do

  1. Drive the development and deployment of ML based search and discovery relevance models and systems integrated with Databricks' products and services.
  2. Design and implement automated ML and NLP pipelines for data preprocessing, query understanding and rewrite, ranking and retrieval, and model evaluation, enabling rapid experimentation and iteration.
  3. Collaborate with product managers and cross-functional teams to drive technology-first initiatives that enable novel business strategies and product roadmaps for the search and discovery experience.
  4. Contribute to building a robust framework for evaluating search ranking improvements - both offline and online.

Skills

Required

  • developing search relevance systems at scale
  • applying LLM to search relevance
  • ML based search and discovery relevance models
  • automated ML and NLP pipelines
  • query understanding and rewrite
  • ranking and retrieval
  • model evaluation
  • search ranking improvements

Nice to have

  • M.S. or PhD preferred
  • Contributions to well-used open-source projects

What the JD emphasized

  • 10+ years experience developing search relevance systems at scale in production or in high-impact research environments.
  • Experience applying LLM to search relevance

Other signals

  • ML based search and discovery relevance models
  • enhancing search ranking
  • improving query understanding
  • building robust evals
  • applying LLM to search relevance