Staff Software Engineer, On-device Hybrid Multimodal AI

Google Google · Big Tech · San Jose, CA +3

Staff Software Engineer role focused on developing and optimizing on-device and hybrid multimodal AI models (LLMs, agents, vision, audio) for XR devices. Responsibilities include leading development, creating evaluation plans, implementing innovations in agent architectures and multimodality, rapid prototyping, and collaborating with research and product teams. Requires strong software development experience with C++, Python, and Generative AI, along with ML/AI experience.

What you'd actually do

  1. Lead the development and optimization of on-device and hybrid multimodal models for XR devices. Utilize techniques to enhance performance and robustness while adhering to strict device power and latency constraints. Write production-quality C++ and Python code.
  2. Create comprehensive evaluation plans for hybrid systems, from dataset development to defining KPIs that measure both model accuracy and on-device efficiency.
  3. Identify, implement, and ship the latest modeling innovations, focusing on hybrid agent architectures, orchestration between edge and cloud, multimodality, tool integrations, and personalization.
  4. Prove out concepts for on-device AI features through rapid prototyping and iterative development, facilitating team testing in close partnership with XR product teams.
  5. Work closely with research scientists, engineers, and product teams to drive the technical roadmap. Foster a collaborative environment and share findings through conference publications while contributing to product launches.

Skills

Required

  • C++
  • Python
  • Generative AI
  • Multimodal Machine Learning
  • deep learning
  • perception
  • computer vision
  • software development

Nice to have

  • JAX
  • TensorFlow
  • PyTorch
  • multimodal learning
  • large language models
  • AI agents
  • prompt engineering
  • few-shot learning
  • post-training techniques
  • evaluations
  • large-scale model training
  • deployment

What the JD emphasized

  • on-device
  • hybrid multimodal models
  • XR devices
  • strict device power and latency constraints
  • production-quality C++ and Python code
  • comprehensive evaluation plans
  • on-device efficiency
  • hybrid agent architectures
  • orchestration between edge and cloud
  • multimodality
  • tool integrations
  • personalization
  • rapid prototyping
  • iterative development
  • technical roadmap
  • conference publications
  • product launches
  • Generative AI
  • Multimodal Machine Learning

Other signals

  • on-device AI
  • multimodal AI
  • LLMs
  • AI agents
  • XR devices