Staff Backline Engineer (spark)

Databricks Databricks · Data AI · Bangalore, India · Support

Databricks is seeking a Staff Backline Engineer to resolve complex customer escalations related to Apache Spark, Delta Lake, and Structured Streaming. This role involves deep code-level analysis, providing best practices, contributing to automation, and acting as a technical bridge between support and engineering. The ideal candidate has extensive experience in developing, testing, and sustaining Python, Java, or Scala applications, with a strong understanding of Spark internals and JVM-based troubleshooting.

What you'd actually do

  1. Troubleshoot, resolve and suggest deep code-level analysis of Spark to address complex customer issues related to Apache Spark™ core internals, Spark SQL, Structured Streaming and Databricks Delta.
  2. Provide best practices guidance around Spark runtime performance and usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers.
  3. Help the support team with detailed troubleshooting guides and runbooks.
  4. Contribute to automation and tooling programs to make daily troubleshooting efficient.
  5. Work with the Spark Engineering Team and spread awareness of upcoming features and releases.

Skills

Required

  • 12+ years of industry experience developing, testing, and sustaining Python or Java or Scala-based applications.
  • Comfortable with compiling, building and navigating the Apache Spark source code.
  • Comfortable with identifying and applying patches/bug fixes to the Apache Spark source code.
  • Experience in Big Data/Hadoop/Spark/Kafka/Elasticsearch data pipelines.
  • Hands-on experience with SQL-based database systems.
  • Experience in JVM, GC, Thread dump-based troubleshooting is required.
  • Experience with AWS or Azure related services.
  • Bachelor's degree in Computer Science or a related field is required.

What the JD emphasized

  • 12+ years
  • developing
  • testing
  • sustaining
  • navigating the Apache Spark source code
  • identifying and applying patches/bug fixes to the Apache Spark source code