Software Engineer - Kafka

Applied Intuition Applied Intuition · Robotics · Sunnyvale, CA · Data Engineering Software Engineering

This role focuses on building the foundational data and machine learning platform for a physical AI company. Responsibilities include data ingestion, processing, storage, labeling infrastructure, and ML infrastructure, handling petabytes of multimodal sensor data. The goal is to support AI research and autonomy stack development, enabling efficient dataset discovery, labeling, and quality assessment.

What you'd actually do

  1. Design and build large-scale data platforms to support our AI research and autonomy stack development, handling petabytes of multimodal sensor data from real-world driving scenarios
  2. Work on data curation and tagging platforms that enable efficient dataset discovery, labeling workflows, and quality assessment across diverse driving conditions
  3. Build high-performance data processing systems using modern distributed computing frameworks to transform raw sensor data into training-ready formats
  4. Use the following technologies: Apache Spark, Apache Hudi, Trino, Apache Kafka, Flyte, Kubernetes, Python, Golang, Java

Skills

Required

  • Bachelor's degree in Computer Science, Software Engineering, or equivalent
  • 2+ years of professional experience
  • Strong backend engineering experience
  • Problem-solving skills
  • experience working with cross-functional teams in fast-paced environments

Nice to have

  • Hands-on experience with modern data stack technologies including Apache Spark, Hudi, Trino, Kafka, or similar distributed data processing frameworks
  • Knowledge of data lake architectures
  • streaming systems
  • workflow orchestration platforms like Flyte

What the JD emphasized

  • petabytes of multimodal sensor data
  • large-scale data platforms
  • high-performance data processing systems

Other signals

  • building foundations of a new workflow
  • massive scope to define how product verticals are deployed at scale
  • 0-1 stage of building up a new team