Lead Software Engineer - Data Platform

JPMorgan Chase JPMorgan Chase · Banking · Austin, TX +1 · Commercial & Investment Bank

Lead Software Engineer for a Data Platform team within Commercial & Investment Banking – Data Analytics – Payments Technology. Responsibilities include designing, building, and maintaining scalable data pipelines, ETL/ELT workflows, and data platform components. The role requires expertise in distributed data processing frameworks, data modeling, and cloud data services, with a focus on delivering secure, high-quality production code in an agile environment.

What you'd actually do

  1. Designs, builds, and maintains scalable data pipelines and ETL/ELT workflows for batch and real-time processing using Spark, Airflow, Kafka, and Flink
  2. Develops data platform components including data cataloging, data quality frameworks, and semantic/metrics layers with embedded governance, lineage, and compliance standards
  3. Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
  4. Implements data modeling strategies (fact and dimensional, wide tables) to support analytics, reporting, and downstream consumption
  5. Partners with analytics teams, product managers, and business stakeholders to translate data requirements into production-grade solutions

Skills

Required

  • software engineering concepts
  • system design
  • application development
  • testing
  • operational stability
  • data platform development
  • Python
  • Java
  • SQL
  • Apache Spark
  • Apache Flink
  • data modeling
  • Apache Airflow
  • AWS S3
  • AWS Glue
  • AWS Redshift
  • AWS Athena
  • AWS EMR
  • AWS Lake Formation
  • Kubernetes
  • Apache Iceberg
  • Unity Catalog
  • OpenMetadata
  • CI/CD
  • Application Resiliency
  • Security

Nice to have

  • Agentic AI
  • LLMs
  • RAG architectures
  • vector databases
  • embedding-based retrieval systems
  • Backstage
  • data mesh
  • data product architectures
  • Terraform
  • Docker
  • data observability
  • data quality
  • metadata management tools
  • semantic layers
  • metrics stores
  • Tableau
  • dbt Metrics