Staff Robotics Software Engineer, Object Perception

Agility Robotics Agility Robotics · Robotics · Fremont +2 · Software Engineering

Senior Robotics Software Engineer focused on object perception for real-world robot manipulation. The role involves owning perception performance end-to-end on hardware, from sensor inputs to how outputs are consumed by downstream planners. It requires working with AI/ML engineers, exploring off-the-shelf models, classical computer vision, geometric reasoning, and learned approaches, including training or fine-tuning models.

What you'd actually do

  1. Own object perception performance on the robot, from sensors through runtime behavior integration
  2. Design, implement, and deploy perception pipelines for manipulation, including object pose estimation, depth reasoning, and scene understanding
  3. Work hands-on with real robots to debug, tune, and harden perception systems under real-world variability
  4. Collaborate with AI/ML teams to evaluate, integrate, train, and fine-tune perception models
  5. Build tools and workflows to evaluate perception performance across datasets, simulation, and hardware

Skills

Required

  • object perception systems for real robotic platforms
  • learned models and classical perception approaches
  • training, fine-tuning, and evaluating perception models
  • production-quality C++ and Python code
  • monocular vision, stereo vision, RGB-D sensing, or depth reconstruction
  • perception libraries (OpenCV, Open3D, PCL, ROS perception, NumPy, scikit-image)

Nice to have

  • Solid understanding of spatial reasoning
  • Demonstrated ability to apply object perception model outputs (detection, segmentation, pose estimation) in manipulation pipelines
  • Experience with structured 3D estimation (plane fitting, normal estimation, voxel maps, etc)

What the JD emphasized

  • 5+ years of professional experience building object perception systems for real robotic platforms
  • Experience working across learned models and classical perception approaches
  • Experience training, fine-tuning, and evaluating perception models
  • Strong experience writing production-quality C++ and Python code
  • Hands-on experience with monocular vision, stereo vision, RGB-D sensing, or depth reconstruction

Other signals

  • object perception
  • real-world robot manipulation
  • production-quality C++ and Python code
  • training or fine-tuning models