Lead Data Engineer

Visa Visa · Fintech · Warsaw, Poland, Poland

Lead Data Engineer responsible for developing next-generation enterprise-level data pipelines and data assets for Real Time and Batch Data ingestion and processing, incorporating Agentic AI to support Visa's 2030 strategy. This role involves overseeing the data lifecycle, leading design and implementation, ensuring data security and compliance, and championing the adoption of GenAI and Agentic AI technologies. The role also includes hands-on development, POCs, establishing engineering best practices, and providing tech leadership and mentoring.

What you'd actually do

  1. Oversee the entire data lifecycle, from data acquisition and ingestion to transformation, storage, and governance for both streaming and batch data pipelines and assets.
  2. Lead the analysis, design, development, and implementation of various initiatives, with end-to-end ownership to deliver business capabilities.
  3. Ensure data security, privacy, governance, and compliance with all relevant regulations, and develop and implement auditable policies and procedures.
  4. Actively contribute with hands on development to critical projects by developing scalable and reusable modules, core frameworks and automation tools.
  5. Champion the adoption of GenAI and Agentic AI technologies and develop strategies to integrate them into existing data pipeline workflows or develop new ones.

Skills

Required

  • 10+ years of work experience with a Bachelor’s Degree or 8+ years of work experience with an Advanced Degree (e.g. Masters/MBA) or 3+ years of work experience with a PhD.
  • Extensive experience as a Data Engineer or in a similar role, with deep expertise in modelling, designing, implementing, deploying, and maintaining complex architectures for Big Data pipelines (ETL, batch, streaming), from inception to delivery.
  • Expertise in at least one programming language (preferably Java or Scala), and one scripting language (preferably Python).
  • Strong experience in large-scale distributed data processing systems using Apache Spark, Kafka, Hadoop, Hive.
  • Experience working with SQL and RDBMS like MySQL or Postgres.
  • Experience in orchestration tools like Apache Airflow.
  • Experience in on-prem and/or cloud-based data platforms (AWS or GCP).
  • Familiarity with Agile methodologies, and modern DataOps / DevSecOps practices.
  • Excellent written and verbal communication skills, interpersonal and presentation skills, with the ability to articulate data concepts to both technical and non-technical audiences. Ability to translate complex business problems into scalable technical solutions.
  • Experience or willingness in GenAI / Agentic engineering (e.g., LLMs, MCPs), and tools such as Cloud Code.

Nice to have

  • Proven ability to lead technical teams, manage multiple projects, and work effectively across geographies and functions.
  • Strong understanding of data quality frameworks, validation methods, and monitoring tools.
  • Experience working with Apache Iceberg, or Delta Tables Databricks.
  • Experience in deployment with automated and scalable CI/CD tools (e.g. Github Actions, Jenkins, Maven), and IaC tools such as Terraform.
  • Experience developing proper metrics instrumentation in Data Platform components using Prometheus, Grafana for operational excellence and incident management.
  • Experience with containerization technologies and orchestration tools, including Docker and Kubernetes.

What the JD emphasized

  • Agentic AI
  • GenAI
  • Agentic engineering

Other signals

  • incorporating Agentic AI to support Visa's 2030 strategy
  • Champion the adoption of GenAI and Agentic AI technologies
  • develop strategies to integrate them into existing data pipeline workflows or develop new ones
  • Experience or willingness in GenAI / Agentic engineering (e.g., LLMs, MCPs)