We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. Join our innovative External Regulatory Financial Control (ERFC)/Strategic Data team at JPMorganChase where we leverage cutting-edge technology to drive data-driven decision-making and enhance business performance. We are seeking a talented and motivated Software/Data Engineer to join our team and contribute to our mission of transforming data into actionable insights.
As a Lead Software Engineer at JPMorgan Chase within the External Regulatory Financial Control (ERFC)/Strategic Data team, you will play a crucial role in designing, developing, and maintaining scalable data processing solutions using Databricks, Python, and AWS. You will collaborate with cross-functional teams to deliver high-quality data solutions that support our business objectives.
Job responsibilities
- Execute creative, data-driven software solutions, including design, development, and technical troubleshooting, with the ability to think beyond routine approaches to solve technical problems.
- Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting.
- Develop fact and dimension data models for reporting and analytics.
- Write secure, high-quality production code, and review and debug code written by others.
- Ensure data quality, consistency, security, and lineage throughout all stages of data processing and transformation, implementing monitoring and alerting mechanisms to maintain pipeline reliability.
- Support data migration and modernization initiatives, transitioning legacy systems to cloud-based data warehouses.
- Lead communities of practice across Software Engineering to promote awareness and adoption of new technologies. Foster a team culture of diversity, opportunity, inclusion, and respect.
- Mentor and guide Associate-level engineers, supporting their technical development and ensuring consistent delivery standards across the team.
- Collaborate with business stakeholders to develop data management strategies, transforming data into insights that drive strategic decisions.
- Document data flows, logic, and transformation rules to maintain transparency and facilitate knowledge sharing.
- Lead and Participate in the full Software Development Life Cycle (SDLC), including requirements gathering, design, development, testing, deployment, and maintenance
- Identify and automate remediation of recurring issues to improve operational stability, acting as both Production Support and SRE function for data pipelines and platform services.
- Utilise AI tools to accelerate development and testing of data pipelines (e.g. GitHub CoPilot, Claude Code).
Required qualifications, capabilities, and skills
- Proven experience in data management, ETL/ELT pipeline development, and large-scale data processing.
- Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning.
- Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).
- Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.).
- Strong understanding of data quality, security, and lineage best practices.
- Experience with cloud-based data warehouse migration and modernization.
- Proficient in CI/CD, continuous delivery methods (Jules/Jenkins, Spinnaker, Sonar), the full Software Development Life Cycle, and Agile methodologies.
- In-depth knowledge of the financial services industry and their IT systems.
- Excellent problem-solving, troubleshooting, and analytical skills with ability to investigate data issues, identify root causes, and implement solutions.
- Strong communication and documentation abilities, with the ability to collaborate effectively with business and technical stakeholders.
Preferred qualifications, capabilities, and skills
- Knowledge of regulatory reporting and financial data aggregation techniques.
- Experience with data orchestration tools (Airflow, Step Functions, etc.).
- Databricks or AWS certifications.
- Demonstrated proficiency in cloud-native development (e.g., cloud, artificial intelligence, machine learning).
- Experience with Parquet, JSON, CSV, Avro, Delta Lake file formats.