Senior Data Engineer

Metropolis Metropolis · Vertical AI · Chicago, IL +5 · Data Engineering & Analytics

Senior Data Engineer role focused on building and optimizing data pipelines, data architecture, and foundational data sets for internal users and members. Requires experience with Python, SQL, ETL/ELT, data modeling, cloud services (AWS), and tools like Snowflake, dbt, and Airflow. Emphasis on data security, privacy compliance, and governance.

What you'd actually do

  1. Collaborate with cross-functional teams to develop end-to-end data pipelines and foundational data sets
  2. Design and own data architecture for large-scale projects while managing operational trade-offs
  3. Build and optimize sophisticated data pipelines, models, and visualizations across multiple domains
  4. Define and manage Service Level Agreements (SLAs) for all owned data sets
  5. Implement data security models, ensure privacy compliance, and evolve data governance processes

Skills

Required

  • Python
  • SQL
  • ETL/ELT
  • data modeling
  • Snowflake
  • dbt
  • RDBMS
  • Airflow
  • Automic
  • CI/CD pipelines
  • IaC ecosystem
  • AWS
  • Glue
  • Airflow (MWAA)
  • DMS
  • EKS
  • SNS

Nice to have

  • Spark
  • PySpark
  • Master's degree in Computer Science, Computer Engineering, or a relevant technical field

What the JD emphasized

  • 5+ years of experience in data engineering
  • 5+ years of experience with Python
  • Hands-on experience with RDBMS
  • working knowledge of CI/CD pipelines
  • Deep understanding of the modern IaC ecosystem
  • hands-on experience with services like Glue, Airflow (MWAA), DMS, EKS, and SNS