Senior Vision Language Model Engineer

NVIDIA NVIDIA · Semiconductors · Santa Clara, CA

Senior Vision Language Model Engineer at NVIDIA to design and build agentic data and training workflows for Autonomous Vehicles, Robotics, and Medical applications. The role focuses on redefining dataset search and model training capabilities within NVIDIA's product offerings, impacting Physical AI developers.

What you'd actually do

  1. Partner with our researchers to develop and evaluate prototypes of our latest models, such as VLMs and VLAs, for video search, video understanding, and more.
  2. Design and implement agentic data workflows that automate data discovery, labeling, evaluation, and retraining to maximize development velocity.
  3. Build, curate, and maintain high‑quality multimodal datasets (e.g., video, sensor, language/action traces) tailored for end‑to‑end physical AI problems, such as autonomous driving.
  4. Explore and productize new data sources including simulation and synthetic data.
  5. Use agentic AI workflows across the full applied research lifecycle.

Skills

Required

  • PhD with 4+ years, MS with 6+ years, or BS (or equivalent experience) with 8+ years of relevant experience in Computer Science, Computer Engineering, or a related technical field
  • Strong background in modern deep learning, including transformer‑based architectures, video modeling, and multimodal VLM/VLA or foundation models.
  • Excellent experience training and deploying deep learning models on real‑world datasets: data preprocessing, distributed training, evaluation, debugging, and iterative improvement.
  • Excellent experience with python and at least one deep learning framework.
  • Current with the latest research on image and video search in autonomous vehicles, healthcare, robotics, or related physical AI applications.
  • Fluent with agentic AI workflows across the full applied research lifecycle, including prototyping novel algorithms and search pipelines, benchmarking, and integrating prototypes in production codebases.
  • Clear and effective communication skills, with experience working well in a dynamic, product- and research-focused team.

Nice to have

  • Strong track record publishing in top-tier conference such as CVPR, NeuRIPS, ICML, ECCV
  • Patents in video retrieval or related field
  • Strong coding architecture skills demonstrated through contributions to large internal or open-source projects.
  • Experience in robotic systems such as autonomous vehicles or humanoid robotics.

What the JD emphasized

  • agentic data workflows
  • agentic AI workflows

Other signals

  • design and build agentic data and training workflows
  • Autonomous Vehicles, Robotics, and Medical applications
  • dataset search platforms for physical AI developers
  • redefine the dataset search and model training capabilities