Principal Software Engineer

JPMorgan Chase JPMorgan Chase · Banking · Jersey City, NJ +1 · Corporate Sector

Principal Software Engineer at JPMorgan Chase focused on designing, implementing, and governing agentic AI systems and LLM-based applications within a regulated financial services environment. The role involves establishing engineering standards for RAG pipelines, embedding workflows, vector store integrations, and model serving, as well as architecting AI-enabled development workflows with guardrails for validation, security, and reuse.

What you'd actually do

  1. Designs and governs agentic AI systems, including multi-agent workflows, tool-use integrations, and human-in-the-loop controls appropriate for regulated financial services environments
  2. Establishes engineering standards for LLM-based applications — RAG pipelines, embedding workflows, vector store integrations, and model serving — ensuring safety, observability, and reproducibility at scale
  3. Architects and governs agentic AI-enabled engineering workflows (using enterprise-authorized tools within the work environment) to improve delivery speed, code quality, and operational outcomes at scale (e.g., AI-driven PR review assistance, test generation/maintenance, release readiness checks, incident triage and root-cause acceleration), while defining guardrails for validation, security, resiliency, and reuse across teams.
  4. Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation at scale.

Skills

Required

  • Formal training or certification on software engineering concepts and 7+ years applied experience
  • Hands-on practical experience delivering system design, application development, testing, and operational stability at enterprise scale
  • Hands-on experience designing and deploying production AI/ML systems, including LLM-based applications and agentic architectures with tool use, memory, and multi-step reasoning in regulated environments
  • Expert in one or more programming languages, particularly Python and/or Java
  • Advanced knowledge of software application development and technical processes, with considerable depth in one or more disciplines (e.g., cloud, AI/ML, data engineering)
  • Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal)
  • Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance
  • Practical cloud-native experience (AWS, Azure, or GCP)
  • Ability to present and effectively communicate with senior leaders and executives
  • Experience in Computer Science, Computer Engineering, Mathematics, or a related technical field
  • Demonstrated experience designing and leading adoption of agentic AI-enabled development practices (using enterprise-authorized tools within the work environment) across teams, including setting standards for human-in-the-loop validation, auditability/traceability of changes, and secure handling of sensitive data.
  • Strong understanding of responsible AI use and control expectations in engineering workflows, including security/resiliency implications, data sensitivity, and risk-based governance; ability to influence senior technical leaders on safe scaling patterns and reuse.

Nice to have

  • Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)
  • Familiarity with AI evaluation and observability practices: evals frameworks, red-teaming, prompt drift detection, and cost/latency monitoring for LLM workloads
  • Understanding of agentic design patterns and how to constrain agent autonomy in high-stakes financial workflows
  • Experience with modern data platforms such as Databricks or Snowflake
  • Deep hands-on experience with Spark/PySpark and other big data processing technologies
  • Expertise in open-source table formats and catalog services such as Apache Iceberg
  • Awareness of AI risk and regulatory considerations relevant to AI use in financial decision-making

What the JD emphasized

  • Hands-on experience designing and deploying production AI/ML systems, including LLM-based applications and agentic architectures with tool use, memory, and multi-step reasoning in regulated environments
  • Demonstrated experience designing and leading adoption of agentic AI-enabled development practices (using enterprise-authorized tools within the work environment) across teams, including setting standards for human-in-the-loop validation, auditability/traceability of changes, and secure handling of sensitive data.
  • Strong understanding of responsible AI use and control expectations in engineering workflows, including security/resiliency implications, data sensitivity, and risk-based governance; ability to influence senior technical leaders on safe scaling patterns and reuse.

Other signals

  • Designing and governing agentic AI systems
  • Establishes engineering standards for LLM-based applications
  • Architects and governs agentic AI-enabled engineering workflows