Senior Data Engineer

Agility Robotics Agility Robotics · Robotics · Fremont +2 · Software Engineering

Senior Data Engineer to join the Data Platform team, responsible for building and maintaining data infrastructure for humanoid robots. This role involves designing, curating, and maintaining high-quality datasets from robot telemetry and logs to enable analytics, debugging, and fleet-scale insights. Responsibilities include developing ETL pipelines, contributing to on-robot data capture, designing derived datasets, implementing data quality checks, and collaborating with various teams.

What you'd actually do

  1. Collaborate with robot software and hardware teams to define, collect, and curate data needed for analytics and debugging.
  2. Develop and maintain ETL pipelines that transform raw robot logs and telemetry into structured datasets using Spark, Airflow (or equivalent orchestration tools), and AWS data services.
  3. Contribute to on-robot data production workflows to ensure high-fidelity, well-structured data capture.
  4. Design derived datasets and transformations across Avro, Parquet, and other sensor data formats to power fleet operations, reliability analysis, and business metrics.
  5. Implement data quality checks, schema evolution, and metadata management practices using our internal Data Registry and cataloging systems.

Skills

Required

  • Apache Spark
  • ETL pipelines
  • AWS data services
  • data modeling
  • schema evolution
  • data quality
  • Python
  • Java
  • Scala
  • AWS S3
  • AWS Glue
  • AWS Athena
  • AWS EMR
  • vision data pipelines

Nice to have

  • robotics vision data
  • C++
  • large-scale robotics data
  • data catalog systems
  • metadata management
  • data versioning
  • Apache Iceberg
  • Delta Lake

What the JD emphasized

  • 5+ years of experience as a Data Engineer or similar role building and maintaining production data pipelines.
  • Strong proficiency in Apache Spark or equivalent distributed data processing frameworks.
  • Experience with Airflow, Dagster, Prefect, or other data orchestration systems.
  • Experience working with vision data pipelines(e.g. Images, video, depth) and building derived datasets from them.