(ind) Senior Manager, Data Engineering

Walmart Walmart · Retail · Chennai, India

Senior Manager, Data Engineering role focused on building and operating a centralized People Data Lake on GCP. Responsibilities include leading a team, owning architecture and delivery of data pipelines (batch and streaming), defining data security frameworks, and partnering with various teams to deliver scalable data solutions. The role emphasizes operational excellence, technical rigor, and continuous improvement, with a focus on data security and compliance.

What you'd actually do

  1. Lead and grow a high-performing team of 8-10 data engineers building and operating the People Data Lake
  2. Own the architecture, design, and delivery of scalable batch and future streaming data pipelines using Spark, Big Query, Airflow, and related cloud-native technologies.
  3. Drive best practices across data modeling, pipeline reliability, observability, and cost optimization to ensure high performance and availability.
  4. Define and implement the data security framework for the lake, including encryption (at rest and in transit), masking and fine-grained access controls.
  5. Establish secure data sharing and governance mechanisms to protect sensitive data and ensure regulatory and enterprise compliance.

Skills

Required

  • 12+ years of experience in software or data engineering
  • 3+ years leading high-performing engineering teams
  • Strong hands-on expertise in Spark, BigQuery, Airflow (or equivalent), and distributed data systems on GCP
  • Experience with big data and distributed systems like Spark, Flink, etc.
  • Proven experience designing, building, and operating large-scale data lakes or lakehouse platforms with batch and streaming ingestion
  • Deep understanding of data security principles including encryption, PII protection, masking, IAM, and row/column-level security
  • Strong SQL and data modeling skills
  • Experience implementing CI/CD, monitoring, and reliability best practices for data platforms
  • Ability to operate at both strategic and execution levels
  • Influence cross-functional stakeholders
  • Translate platform vision into secure, scalable business outcomes
  • Knowledge in Pub sub system like Kafka
  • Knowledge of cloud platforms like Azure, GCP etc

Nice to have

  • Lead the integration of AI/ML toolkits and frameworks into day-to-day (D2D) data engineering workflows

What the JD emphasized

  • Data Security charter
  • Data Security
  • encryption
  • masking
  • fine-grained access controls
  • regulatory and enterprise compliance