Staff Data Engineer, Data Engineering

Visa Visa · Fintech · Bangalore, India

Staff Data Engineer responsible for designing, building, and optimizing data pipelines and platforms for Visa's analytics and decisioning capabilities. The role involves architecting and implementing large-scale data pipelines and modern data warehouse solutions, providing technical leadership, and leveraging Generative AI to enhance the data engineering lifecycle.

What you'd actually do

  1. Build and utilize data modeling frameworks and leverage modern architectural patterns such as Medallion Architecture to design scalable, well-structured, and high-performance data solutions.
  2. Define, execute, and manage large-scale data pipelines to build solutions with required scalability and flexibility for global delivery.
  3. Build and maintain ETL processes in Spark, Python, and Hive, ensuring data quality, testing, and standardization across multiple sources.
  4. Develop modular and reusable data pipeline components to reduce code duplication and ensure consistency across projects.
  5. Enhance pipeline performance and reliability through continuous monitoring, proactive refactoring, and integration of modern frameworks and tools.

Skills

Required

  • Data modeling frameworks
  • Medallion Architecture
  • Large-scale data pipelines
  • ETL processes
  • Spark
  • Python
  • Hive
  • Data quality
  • Testing
  • Standardization
  • Modular and reusable data pipeline components
  • Pipeline performance and reliability enhancement
  • Technical/data documentation
  • Data dictionaries
  • Code version control (Git)
  • Visualization dashboards (Tableau or Power BI)
  • Generative AI (GenAI) for data engineering
  • Requirement gathering
  • Collaboration with business and technical teams
  • Communication of technical insights
  • Best practices for maintainability and scalability
  • Coding standards
  • Automated workflows
  • Technical leadership
  • Mentorship

What the JD emphasized

  • scalable
  • high-quality
  • large-scale data pipelines
  • modern data warehouse solutions
  • technical leadership
  • advanced data modeling frameworks
  • modern architectural patterns
  • Medallion Architecture
  • well-structured
  • efficient
  • future-ready data solutions
  • scalability
  • flexibility
  • global delivery
  • data quality
  • testing
  • standardization
  • modular
  • reusable
  • reduce code duplication
  • consistency
  • pipeline performance
  • reliability
  • continuous monitoring
  • proactive refactoring
  • modern frameworks and tools
  • technical/data documentation
  • data dictionaries
  • validation alerts
  • code version control
  • Git
  • data accuracy
  • data integrity
  • lightweight visualization dashboards
  • analytical insights
  • Generative AI (GenAI)
  • data engineering lifecycle
  • automating code generation
  • improving documentation
  • optimizing pipeline design
  • requirement gathering
  • global business and technical teams
  • smooth solution development
  • communicating technical insights and recommendations
  • stakeholders and leadership
  • best practices for maintainability and scalability
  • coding standards
  • version control
  • automated workflows
  • technical leadership and mentorship
  • team of data engineers
  • promoting best practices
  • delivery of high-quality, scalable solutions