What you'd actually do

Design, implement, and manage the data architecture, systems, and processes to effectively collect, store, process and analyze high volume, high dimensional data to provide strategic insight into complex business problems.

Create and maintain scalable, efficient, and secure data pipelines, data warehouses, and data lakes.

Ensure consistency in data quality and availability for analysis and reporting including compliance with data governance and security standards.

Work in complex data science and analytics projects in support of the Value Engineering organization.

Develop and maintain ETL (Extract, Transform, Load) processes.

Skills

Required

Data Engineering
ETL Development
Database Administration
Azure Data Factory
Azure DW
Databricks
APIs
Python
SQL
PySpark
Structured data
Snowflake
Agile development methodology

Nice to have

Data Mining concepts
Classification
Predictive algorithms
Data integration tools
Data Modelling techniques
schema design
polars
scikit
technical vision

Advanced Data Engineer

Location: Bangalore (Hybrid)

Join a team that is elevating our strategy to drive advanced analytics and visualization tools across the Value Engineering COE. In this role, Advanced Data Engineer , you will design, implement, and manage the data architecture, systems, and processes to effectively collect, store, process and analyze high volume, high dimensional data to provide strategic insight into complex business problems. This will involve creating and maintaining scalable, efficient, and secure data pipelines, data warehouses, and data lakes. You need to ensure consistency in data quality and availability for analysis and reporting including compliance with data governance and security standards.

Duties and Responsibilities

Work in complex data science and analytics projects in support of the Value Engineering organization.
Understand and translate business requirements into technical solutions, leveraging available and new technologies as required.
Design and implement data models and schemas to support analytical and reporting requirements.
Collaborate with data scientists and data analysts to define and structure data for effective analysis and reporting.
Develop and maintain ETL (Extract, Transform, Load) processes.
Administer, optimize, and manage databases, data warehouses, and data lakes to ensure performance, reliability, and scalability.
Enforce data governance policies, standards, and best practices to maintain data quality, privacy, and security.
Create and maintain comprehensive documentation for data architecture, processes, and systems.
Troubleshoot and resolve data-related problems and optimize system performance.
Partner with IT support team on production processes, continuous improvement, and production deployments.

YOU MUST HAVE

7 plus years of relevant experience in Data Engineering, ETL Development, Database Administration.
Experience in Azure Data Factory, Azure DW, Databricks, and APIs
Expertise in querying and programming languages, such as Python, SQL, PySpark
Experience with Structured data. Knowledge of unstructured data is not needed but valued.
Experience in Snowflake
Knowledge of Agile development methodology

WE VALUE

Knowledge of databases, data warehouse platforms (Snowflake) and Cloud based tools.
Understanding of Data Mining concepts & applications such as Classification or Predictive algorithms.
Experience in using data integration tools for ETL processes.
Knowledge of Data Modelling techniques including schema design for relational databases
Experience with Python packages such as: polars, scikit – especially those packages linked to data processing & transformation.
Ability to develop and communicate technical vision for projects and initiatives that can be understood by customers and management.