Lead Data Engineer (python, Aws, Sql, Genai) (enterprise Platforms Technology)

Capital One Capital One · Banking · McLean, VA

Lead Data Engineer role focused on building data pipelines and platforms for marketing and messaging personalization, utilizing Python, SQL, and cloud technologies like AWS, Snowflake, and Databricks. The role involves collaborating with Agile teams, optimizing information systems, and mentoring engineers. While GenAI is mentioned, the core responsibilities are data engineering and pipeline development.

What you'd actually do

  1. Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies
  2. Work, as a lead developer, with a team of developers with deep experience in data movement, distributed computing, and full stack systems
  3. Utilize programming languages like Python, SQL and Open Source RDBMS and NoSQL databases and Cloud based data warehousing services such as Snowflake, Databricks
  4. Optimize information system for end-user and downstream application consumers by using sound data design practices
  5. Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal & external technology communities, and mentoring other members of the engineering community

Skills

Required

  • Python
  • SQL
  • AWS
  • Snowflake
  • Databricks
  • data modeling
  • data pipelines
  • big data technologies
  • cloud computing

Nice to have

  • Spark
  • ETL tools
  • AWS Glue
  • MapReduce
  • Hadoop
  • Hive
  • EMR
  • Kafka
  • MySQL
  • real-time data
  • streaming applications
  • NoSQL implementation
  • Mongo
  • Cassandra
  • data warehousing
  • Redshift
  • UNIX/Linux
  • shell scripting
  • Agile engineering practices
  • AI-assisted coding tools

What the JD emphasized

  • GenAI