AI Research Engineer - Applied Scientist Compilers

NVIDIA NVIDIA · Semiconductors · Santa Clara, CA +5 · Remote

AI Research Engineer/Applied Scientist focused on Compilers/Low-level optimization to develop AI compiler solutions for NVIDIA's software stack and GPU acceleration. Responsibilities include applying AI to compilation, implementing AI-based solutions for GPU programming, building training pipelines (fine-tuning, RL), defining model I/O, developing evaluation frameworks, prompt engineering, integrating learned policies, prototyping models, creating datasets, and applying RL for optimization.

What you'd actually do

  1. Help trailblaze company efforts in applying AI within conventional compilation pipelines.
  2. Design and implement AI-based technology addressing core problems of low-level GPU programming.
  3. Build training pipelines for supervised fine-tuning and reinforcement learning (RL/RLHF-style or policy optimization variants).
  4. Define model inputs/outputs over compiler low level compiler representations.
  5. Develop evaluation frameworks to measure code quality, runtime, compile-time overhead, and correctness.

Skills

Required

  • M.S./PhD degree in Computer Engineering, Computer Science related technical field (or equivalent experience)
  • 5+ years of experience building AI/ML systems
  • Strong software engineering skills in Python
  • Strong software engineering skills in C++
  • Hands-on experience training/fine-tuning large models (Transformers, PEFT/LoRA, distributed training)
  • Solid understanding of machine learning fundamentals and experimentation best practices
  • Experience with reinforcement learning (e.g., policy gradients, actor-critic, offline RL, bandit-style optimization)
  • Knowledge of prompt-engineering techniques
  • Ability to work across research and engineering, from prototype to production

Nice to have

  • Distributed training/inference at scale
  • Experience working with the NVIDIA NeMo framework
  • Understanding of GPU performance
  • Experience with benchmarking suites and performance profiling tools
  • Formal methods or static analysis familiarity for correctness guarantees
  • CUDA programming experience

What the JD emphasized

  • 5+ years of experience building AI/ML systems
  • Hands-on experience training/fine-tuning large models
  • Experience with reinforcement learning
  • Ability to work across research and engineering, from prototype to production

Other signals

  • applying AI within conventional compilation pipelines
  • AI-based technology addressing core problems of low-level GPU programming
  • training pipelines for supervised fine-tuning and reinforcement learning
  • integrate learned policies into production toolchains
  • Apply RL techniques to optimize for downstream objectives