Senior Software Engineer - GPU Kernel Authoring & Optimization

Weights & Biases Weights & Biases · Data AI · Bellevue, WA +1 · Technology

Senior Software Engineer focused on authoring and optimizing GPU kernels for large-scale LLM inference serving. The role involves deep understanding of GPU architecture, CUDA programming, and performance benchmarking to achieve maximum throughput and minimum latency. Responsibilities include kernel development, optimization, benchmarking, and contributing to the inference stack's performance and reliability.

What you'd actually do

  1. Author, profile, and optimize CUDA kernels—GEMMs, attention, MoE routing, quantization, KV-cache, and fused epilogues—on the critical path of LLM inference.
  2. Optimize for the hardware: exploit tensor cores and tune occupancy, memory coalescing, shared-memory/register usage, and overlap of compute with data movement.
  3. Benchmark rigorously: build reproducible microbenchmarks and roofline analyses, and validate that kernel-level wins translate to end-to-end latency/throughput gains across model-serving stacks (vLLM, TensorRT-LLM, llm-d, SGLang).
  4. Implement and maintain benchmarking workflows for end-to-end MLPerf Inference (and Training) runs, including workload setup, cluster configuration, runbooks, and result validation.
  5. Lead design reviews and drive architecture within the team; decompose multi-service work into clear milestones.

Skills

Required

  • 5+ years of experience building high-performance computing, GPU/accelerator software, or performance-critical systems.
  • Hands-on CUDA experience
  • Deep understanding of GPU architecture and performance
  • Strong coding in C++ and Python
  • Familiarity with model-serving stacks (vLLM, TensorRT-LLM, llm-d, SGLang)

Nice to have

  • Triton or Mojo for authoring custom GPU kernels
  • CuTe DSL for Python-based kernel authoring on NVIDIA GPUs
  • JAX and its Pallas kernel language
  • HIP / ROCm and AMD GPU experience
  • NCCL and collective-communication performance
  • Experience with alternative accelerators such as Google TPUs and Meta's MTIA
  • Familiarity with kernel-authoring DSLs and nano-compilers such as KNYFE and its Block DSL
  • Experience with Kubernetes at production scale
  • Experience with SUNK (Slurm on Kubernetes) / Slurm for scheduling large GPU jobs
  • Experience running MLPerf submissions or similar large-scale audited benchmarks
  • Contributions to OSS projects such as vLLM, SGLang, PyTorch, Triton, or CUTLASS

What the JD emphasized

  • Hands-on CUDA experience is required
  • Deep understanding of GPU architecture and performance
  • Author, profile, and optimize CUDA kernels
  • Benchmark rigorously
  • MLPerf Inference (and Training) runs

Other signals

  • GPU kernel optimization
  • LLM inference performance
  • MLPerf benchmarking