Member of Technical Staff - Data Engineer

Microsoft Microsoft · Big Tech · New York, NY +4 · Data Engineering

Microsoft AI is seeking an experienced Data Engineer to build mission-critical data pipelines for their Copilot systems. The role involves ingesting, processing, and publishing data streams to support AI model fine-tuning and introspection. The ideal candidate will have strong big data processing skills and experience with cloud infrastructure.

What you'd actually do

  1. Build scalable data pipelines for sourcing, transforming and publishing data assets for AI use cases.
  2. Work collaboratively with other Platform, infrastructure, application engineers as well as AI Researchers to build next generation data platform products and services.
  3. Ship high-quality, well-tested, secure, and maintainable code.
  4. Find a path to get things done despite roadblocks to get your work into the hands of users quickly and iteratively.
  5. Enjoy working in a fast-paced, design-driven, product development cycle.

Skills

Required

  • Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling or data engineering work OR Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, or data engineering work OR equivalent experience.

Nice to have

  • 4+ years technical engineering experience building data processing applications (batch and streaming) with coding in languages including, but not limited to, Python, Java, Spark, SQL.
  • Experience working with Apache Hadoop eco system, Kafka, NoSQL, etc.
  • 3+ years experience with data governance, data compliance and/or data security.
  • 2+ years' experience building scalable services on top of public cloud infrastructure like Azure, AWS, or GCP. Extensive use datastores like RDBMS, key-value stores, etc.
  • 2+ years' experience building distributed systems at scale and extensive systems knowledge that spans bare-metal hosts to containers to networking.
  • Ability to identify, analyze, and resolve complex technical issues, ensuring optimal performance, scalability, and user experience.
  • Dedication to writing clean, maintainable, and well-documented code with a focus on application quality, performance, and security.
  • Demonstrated interpersonal skills and ability to work closely with cross-functional teams, including product managers, designers, and other engineers.
  • Ability to clearly communicate complex technical concepts to both technical and non-technical stakeholders.
  • Interest in learning new technologies and staying up to date with industry trends, best practices, and emerging technologies in web development and AI.
  • Ability to work in a fast-paced environment, manage multiple priorities, and adapt to changing requirements and deadlines.

What the JD emphasized

  • mission critical data pipelines
  • build core data pipelines that help fine tune models
  • technical engineering experience building data processing applications (batch and streaming)
  • data governance, data compliance and/or data security

Other signals

  • build mission critical data pipelines
  • ingest, process and publishes data streams from our personal AI, Copilot systems
  • build core data pipelines that help fine tune models, support introspection and retrospection of data