Sr Data Engineer

Disney Disney · Media · Santa Monica, CA +4

Senior Data Engineer responsible for designing, building, and optimizing data pipelines and transformation frameworks for acquisition reporting and marketing analytics. This role involves working with AWS, Databricks, Unity Catalog, Snowflake, and Airflow to ingest and model marketing platform data, ensuring data accuracy and reliability for downstream dashboards. Collaboration with product, analytics, and marketing teams is key.

What you'd actually do

  1. Architect, build, and maintain scalable ETL/ELT pipelines for acquisition reporting using Databricks, PySpark, SQL, and Unity Catalog.
  2. Lead the modernization effort to migrate existing Snowflake‑based SQL scripts and transformations into Databricks UC with best practices in governance and structured access.
  3. Design robust ingestion frameworks for marketing vendor data.
  4. Implement data quality checks, monitoring, and automated remediation using Databricks, Snowflake, Airflow, and internal frameworks.
  5. Develop metadata-driven, parameterized pipeline components to accelerate onboarding of new vendors and datasets.

Skills

Required

  • SQL
  • PySpark
  • Spark SQL
  • Data Modeling
  • ETL/ELT design patterns
  • Distributed data processing
  • Databricks
  • Delta Lake
  • Unity Catalog
  • AWS
  • S3
  • IAM
  • EC2
  • Glue
  • Lambda
  • Airflow
  • JSON
  • Parquet
  • CSV
  • Git/GitHub
  • CI/CD
  • DevOps practices
  • Communication skills

Nice to have

  • Master’s degree in computer science, Information Systems or related field
  • Marketing or customer acquisition data (Meta, Google Ads, Google CM360, TikTok, Twitter, Snapchat, Branch, AppsFlyer, Salesforce, etc.)
  • Data observability
  • SLA monitoring
  • Incident workflows
  • Reliability engineering concepts
  • Great Expectations
  • Deequ
  • Monte Carlo

What the JD emphasized

  • 5+ years of experience as a Data Engineer or similar role
  • Strong proficiency in SQL
  • Hands-on experience with PySpark and/or Spark SQL in production
  • Strong understanding of data modeling, ETL/ELT design patterns, and distributed data processing
  • Experience building pipelines in Databricks, including Delta Lake, Unity Catalog, data governance, and Lakehouse patterns
  • Strong experience in AWS
  • Proficiency with Airflow or similar orchestration tools
  • Experience building robust ingestion pipelines and working with semi‑structured formats
  • Experience with Git/GitHub, CI/CD, and modern DevOps practices
  • Bachelor’s degree in computer science, Information Systems or related field