Senior Software Engineer - Data Infrastructure

Applied Intuition Applied Intuition · Robotics · Sunnyvale, CA · Data Engineering Software Engineering

This role focuses on scaling open-source data infrastructure for a company that provides digital infrastructure for physical AI. The engineer will work across the entire data lifecycle, from collection to retrieval, and design/develop external and internal products. The core responsibility is handling massive data volumes and supporting data products and verticals.

What you'd actually do

  1. Scale infrastructure to support all deployment types (cloud, hybrid, on-prem) and across regions
  2. Be involved in the end to end data lifecycle, from the external-facing product to the underlying platform and infrastructure for it
  3. Build features to tune processing pipeline for fast data ingestion and indexing depending on customer's needs and workloads
  4. Enable product workflows that expose performant query interfaces and offer easy-to-use integration hooks
  5. Develop and deploy high-quality software using modern tooling and frameworks, especially open-source technologies

Skills

Required

  • Experience with large-scale open source data technologies (Spark, Kafka, Hudi, Flyte, etc.)
  • Experience with containerization and other modern software development workflows
  • Knowledge of the open source landscape with judgment on when to choose open source versus build in-house
  • 3+ years of professional experience
  • Bachelor's degree in Computer Science, Software Engineering, or equivalent

Nice to have

  • Expertise with modern programming languages (Python, C++, GoLang, Scala, etc.)
  • Experience with other open-source data technologies not listed above
  • Expertise with Kubernetes
  • Experience with enterprise software, including on-prem and/or cloud environments
  • Deep knowledge of data quality, data profiling and cleansing techniques

What the JD emphasized

  • Handling massive volumes of data
  • scale open-source data infrastructure