Senior Data Engineer

Walmart Walmart · Retail · Bentonville, AR

Senior Data Engineer responsible for understanding and applying data strategy principles, identifying data sources, performing data quality checks, creating data pipelines, transforming data, and developing data models. The role involves translating business problems into data solutions, developing business cases, and creating training documentation. It also includes code development, testing, deployment, and contributing to data governance practices.

What you'd actually do

  1. Understands, articulates, and applies principles of the defined strategy to routine business problems that involve a single function.
  2. Supports the understanding of the priority order of requirements and service level agreements.
  3. Extracts data from identified databases.
  4. Creates data pipelines and transforms data to a structure that is relevant to the problem by selecting appropriate techniques.
  5. Translates/co-owns business problems within one's discipline to data related or mathematical solutions.

Skills

Required

  • designing, building, installing, configuring, and supporting automated ETL pipelines using various programming languages and tools, including Spark or Scala, Unix Shell Scripts, HQL, Hadoop, Hive, Cloud Platform, and Automic scheduler
  • building and scheduling ETL pipelines, Aorta ETL framework, and Spark framework
  • working with multiple business areas to capture detailed requirements to build software solutions and new data sources to add to the data lake platform centralized repository
  • performing data analysis
  • building Data Models based on data availability and business requirements using CA Erwin tool
  • relational and non-relational database management systems such as DB2, IMS, SQL Server, Hive, Informix, Cassandra, Cosmos DB, Solr, and Teradata
  • developing, testing, installing, and supporting software solutions using mainframe technologies such as COBOL, Easytrieve, JCL, VSAM, DB2, and CA7
  • documenting technical solution design to capture detailed information about the project’s technology stack and integration with interfacing software systems
  • designing and developing data migration solutions to load data from on-prem servers and databases to Cloud platform
  • working with tools and utilities such as SFTP, Alteryx, QMF, DBeaver, WinSCP, FileZilla, Jira, and ServiceNow

What the JD emphasized

  • Master’s degree or the equivalent in Computer Science, or a related field plus 1 year of experience in software engineering or related experience; OR Bachelor's degree or the equivalent in Computer Science, or a related field plus 3 years of experience in software engineering or related experience.