Sr Data Engineer

Disney Disney · Media · New York, NY +2

Seeking a Senior Software Engineer to join the Product Performance Team within the Data Organization at Disney Entertainment and ESPN. This role focuses on designing, building, and testing core algorithms and data models for reliable data solutions that measure product performance (browse, playback attribution, search) for Disney+. Responsibilities include developing batch and stream processing data solutions using Scala and Python with frameworks like Spark, establishing data quality and governance standards, evolving data models, and collaborating with cross-functional teams.

What you'd actually do

  1. Design, implement, and deploy robust data solutions for both batch ingestion and stream processing using Scala and Python with frameworks like Spark, primarily deployed via Databricks and Airflow.
  2. Establish and enforce rigorous standards for code development, testing, and deployment to guarantee the highest levels of data quality, accuracy, and governance.
  3. Extend functionality of Product Performance Data offerings by evolving existing data models and developing new data pipelines to introduce novel metrics and dimensions critical for measuring Disney+ engagement.
  4. Maintain, update, and expand existing source code that powers core datasets, ensuring strict quality standards and high data reliability.
  5. Drive platform and business success through active collaboration with Product Managers, Data Analysts, and cross-functional engineering teams to define requirements and deliver new data features.

Skills

Required

  • Scala
  • Python
  • Spark
  • Databricks
  • Airflow
  • SQL
  • Kafka
  • CICD
  • SDLC
  • Bash
  • PowerShell
  • AWS

Nice to have

  • Snowflake
  • Redshift
  • Big Query
  • Jenkins
  • GitHub Actions
  • GraphQL
  • Data Modeling
  • Data Warehousing
  • Scrum
  • Agile

What the JD emphasized

  • 5+ years of software engineering experience developing large data pipelines
  • Strong CICD and SDLC fundamentals
  • Strong algorithmic problem-solving expertise
  • Strong fundamental Scala and Python software engineering skills
  • Hands-on production environment experience with distributed engineering systems such as Spark and Kafka
  • Hands-on production experience with orchestration systems such as Airflow