Software Engineer, Evaluation Infrastructure

Waabi Waabi · Robotics · Toronto, ON +2 · Remote · Autonomy & Algorithms

Software Engineer focused on building evaluation infrastructure and tooling for an autonomous driving system. This involves developing pipelines, implementing metrics, and creating query services for analyzing driving performance data from real-world and simulated sources, leveraging generative AI in the simulation engine.

What you'd actually do

  1. Develop the tooling, infrastructure, and pipelines to support complex statistical analyses of driving performance scale.
  2. Implement metrics and tags to provide a holistic understanding of model performance and enable the discovery of interesting scenarios for training and evaluation.
  3. Develop and maintain a high availability query service to enable low-latency analysis and curation over large volumes of metric and tag data
  4. Work with large datasets from various sources including real world driving as well as Waabi World, our high-fidelity simulator.
  5. Champion engineering excellence, ensuring high-quality, well structured and tested code.

Skills

Required

  • Python programming
  • strong software engineering fundamentals
  • high quality, well-structured, and well-tested code

Nice to have

  • data processing pipelines
  • ETL pipelines
  • distributed computing
  • highly reliable and scalable web services
  • cloud job orchestration
  • monitoring
  • instrumentation best-practices
  • evaluating complex ML models or self-driving software stacks

What the JD emphasized

  • evaluating complex ML models or self-driving software stacks

Other signals

  • evaluating autonomy system performance
  • simulation engine built with generative AI
  • metrics and tags for model performance
  • large datasets from real world driving and simulator