Distinguished Data Engineer - Enterprise Data Technology

Capital One Capital One · Banking · McLean, VA +1

Distinguished Data Engineer role focused on designing and implementing scalable data architectures, optimizing data pipelines, and leading the development of agentic workflows within an enterprise data ecosystem. The role involves integrating AI/ML into operations for predictive and automated platform capabilities, with a strong emphasis on Databricks, Snowflake, and AWS.

What you'd actually do

  1. Lead the design and implementation of highly scalable, fault-tolerant, and cost-effective data architectures that seamlessly integrate Databricks (for complex processing,ML) and Snowflake (for warehousing,ETL).
  2. Drive performance engineering and optimization for large-scale data ingestion and processing workloads across the Databricks,Snowflake,AWS data pipeline.
  3. Provide technical leadership for the design, development, testing and deployment of agentic workflows across Capital One.This position requires a combination of strategic thinking, technical expertise in AI,ML and strong leadership to align the platform with business goals
  4. Deep technical experts and thought leaders that help accelerate adoption of the very best engineering practices, while maintaining knowledge on industry innovations, trends and practices
  5. Evangelists, both internally and externally, helping to elevate the Distinguished Engineering community and establish themselves as a go-to resource on given technologies and technology-enabled capabilities

Skills

Required

  • data engineering
  • data architecture
  • AWS
  • Python
  • SQL
  • Scala
  • AI and ML algorithms or technologies

Nice to have

  • Databricks
  • Snowflake
  • Data Governance
  • LLM Inference
  • Similarity Search
  • VectorDBs
  • Guardrails
  • Memory
  • C++
  • C#
  • Java
  • Golang
  • Delta Lake
  • Unity Catalog
  • MLflow
  • Spark workloads
  • Data Sharing
  • Snowpipe
  • external tables
  • security features
  • cost governance
  • advanced SQL
  • Stored Procedures

What the JD emphasized

  • agentic workflows
  • AI,ML
  • technical expertise in AI,ML
  • deploying scalable and responsible AI solutions

Other signals

  • integrating AI,ML directly into our operations
  • transforming our model from reactive maintenance to a predictive, self-healing, and automated platform architecture
  • design and implementation of highly scalable, fault-tolerant, and cost-effective data architectures
  • Drive performance engineering and optimization for large-scale data ingestion and processing workloads
  • technical leadership for the design, development, testing and deployment of agentic workflows