Data Engineer II

Microsoft Microsoft · Big Tech · Redmond, WA +1 · Data Engineering

Data Engineer II role focused on maintaining and advancing a data platform using Spark, PySpark, and Spark SQL within Microsoft Fabric. The role involves code optimization, metadata-driven orchestration, DevOps best practices, and providing feedback on new products and features. It also includes live site and on-call duties for analytics solutions.

What you'd actually do

  1. Extract and transform disparate data into actionable insights that will drive business decisions. Applying industry best practice transformation and modeling techniques – validating the output to ensure reliable and accurate results for consumption by downstream data teams, and end users.
  2. Drive development projects to evolve our platform and operations, leveraging the newest Microsoft technologies available to us.
  3. Contribute to the success of Azure Data by trying out and testing new products and features, contributing to the design of major feature enhancements, and dedicating time to provide direct feedback to the Product Group.
  4. Work directly with business stakeholders to refine requirements, iterate and finalize design, deliver working proofs of concept, and develop final data solution.
  5. Live site and on call DRI duty for maintaining high availability of our analytics solutions .

Skills

Required

  • Spark development
  • PySpark
  • Spark SQL
  • Microsoft Fabric
  • DevOps
  • GIT operations
  • CICD flows
  • Azure DevOps
  • time management
  • prioritization
  • written communication
  • verbal communication

Nice to have

  • Microsoft Fabric
  • Azure Data Factory
  • Azure Synapse
  • Azure Databricks
  • GIT operations
  • CICD flows
  • Azure DevOps
  • time management
  • prioritization
  • written communication
  • verbal communication
  • passion for data
  • desire to learn & adopt new technologies