Sr Databricks Data Engineer

This role focuses on designing, building, and optimizing scalable data engineering solutions using Databricks across cloud environments. It involves creating enterprise data pipelines, data models, and cloud-based data architectures, as well as building and optimizing batch and streaming data solutions. The role also involves advising stakeholders on platform capabilities and driving engineering quality through automation and CI/CD practices. While the team supports AI/ML use cases, the core function is data engineering.

What you'd actually do

  1. Lead the design and implementation of enterprise data pipelines, data models, and cloud-based data architectures using Databricks
  2. Build and optimize batch and streaming data solutions that support analytics, reporting, and operational use cases
  3. Advise stakeholders on platform capabilities, architecture decisions, governance controls, and delivery approaches aligned to business objectives
  4. Drive engineering quality through automation, continuous integration/continuous deployment (CI/CD), performance tuning, and scalable delivery practices
  5. Contribute to technical best practices, evaluate emerging platform capabilities, and support teams delivering modern data engineering solutions

Skills

Required

  • Databricks
  • AWS, Azure, or GCP
  • Apache Spark
  • Delta Lake
  • Lakehouse architecture
  • enterprise data warehousing
  • PySpark
  • Delta Live Tables
  • Autoloader
  • Structured Streaming
  • Databricks Workflows
  • Databricks Unity Catalog
  • CI/CD pipeline deployment

Nice to have

  • Master’s degree
  • Databricks Lakeflow
  • supporting artificial intelligence or machine learning use cases