Lead Software Engineer, Java/spark/aws

JPMorgan Chase JPMorgan Chase · Banking · Jersey City, NJ +1 · Corporate Sector

Lead Software Engineer role focused on designing, building, and optimizing ETL data pipelines using Databricks and AWS services. The role involves architecting data lake and data warehouse solutions, collaborating with stakeholders, mentoring engineers, and ensuring operational stability. While AI tools for developer productivity are mentioned, the core function is data engineering and software development.

What you'd actually do

  1. Design, build, maintain and optimize robust ETL data pipelines using Databricks (Spark, Delta Lake, Unity Catalog) and ensure efficient ingestion, transformation, and storage
  2. Collaborate with data product owner, business stakeholders, and ensure best practices in data engineering, software engineering and resilient cloud architecture
  3. Architect and implement data lake and data warehouse solutions leveraging AWS services (S3, Glue, SQS, SNS, Lambda, EMR, etc.)
  4. Collaborate with cross team to propose and build new solution for supporting overall application platform by the means of observability, orchestration, resiliency, developer experience, automation.
  5. Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems

Skills

Required

  • Formal training or certification on software engineering concepts and 5+ years applied experience
  • 5+ years of hands-on professional experience in one or more programming language(s), including Java or Python for data engineering tasks
  • Hands-on experience with Databricks (Spark, Delta Lake, notebooks, job orchestration), AWS data services(EMR, Athena, Glue, S3)
  • Hands-on experience utilizing Apache Spark for large-scale data processing, including developing and optimizing data pipelines, performing real-time and batch analytics, and leveraging Spark’s libraries for machine learning and data transformation to drive actionable business insights.
  • Experience of leveraging AI tools for Developer productivity increase (SDD, Agentic AI, Skills, Copilot, Claude Code etc.)
  • Experience with modern monitoring and logging tools (e.g. Dynatrace, Splunk, Grafana, Prometheus).
  • Proficiency in all aspects of the Software Development Life Cycle with familiarity on CI/CD, DevOps, and infrastructure-as-code tools in cloud environments
  • Proven leadership experience in leading and mentoring varying levels of software engineers

Nice to have

  • Application development experience in delivering complicated enterprise Investment Banking application for Market Surveillance, or Investment Banking Front-office Trading Systems or Analytics Systems in FX, Commodities, Equities and Equities Derivatives domains
  • Familiarity with SpringBoot based microservices architecture and RESTful API development.
  • Experience in Container technologies (i.e. Kubernetes and Docker)
  • Experience in Kafka streaming
  • Financial Products knowledge of Futures & Options, FX, Commodities, Equities and Equities Derivatives, as well as trade lifecycles and/or order workflow

What the JD emphasized

  • 5+ years of hands-on professional experience in one or more programming language(s), including Java or Python for data engineering tasks
  • Hands-on experience with Databricks (Spark, Delta Lake, notebooks, job orchestration), AWS data services(EMR, Athena, Glue, S3)
  • Hands-on experience utilizing Apache Spark for large-scale data processing, including developing and optimizing data pipelines, performing real-time and batch analytics, and leveraging Spark’s libraries for machine learning and data transformation to drive actionable business insights.