Machine Learning Performance Engineer

Jane Street Jane Street · Quant · London, United Kingdom · Machine Learning

Machine Learning Performance Engineer role focused on optimizing the performance of ML models for both training and inference. Requires deep low-level systems programming and GPU knowledge, debugging skills, and experience with ML libraries and distributed training.

What you'd actually do

  1. optimising the performance of our models – both training and inference
  2. improving straightforward CUDA, but the interesting part needs a whole-systems approach, including storage systems, networking and host- and GPU-level considerations
  3. ensure our platform makes sense even at the lowest level – is all that throughput actually goodput?
  4. debug a training run’s performance end to end
  5. improve the performance of our models

Skills

Required

  • Experience in low-level systems programming and optimisation
  • Experience debugging ML model performance end to end
  • Low-level GPU knowledge (PTX, SASS, warps, cooperative groups, Tensor Cores, memory hierarchy)
  • Debugging and optimisation experience using tools like CUDA GDB, NSight Systems, NSight Computesight-systems and nsight-compute
  • Library knowledge of Triton, CUTLASS, CUB, Thrust, cuDNN and cuBLAS
  • Understanding of latency and throughput characteristics of CUDA graph launch, tensor core arithmetic, warp-level synchronization and asynchronous memory loads
  • Background in Infiniband, RoCE, GPUDirect, PXN, rail optimisation and NVLink
  • Understanding of collective algorithms supporting distributed GPU training in NCCL or MPI
  • Fluency in English

Nice to have

  • An understanding of modern ML techniques and toolsets
  • An inventive approach and the willingness to ask hard questions about whether we're taking the right approaches and using the right tools

What the JD emphasized

  • low-level systems programming and optimisation
  • low-latency inference
  • high-throughput inference
  • low-level GPU knowledge
  • debugging and optimisation experience
  • Library knowledge of Triton, CUTLASS, CUB, Thrust, cuDNN and cuBLAS
  • networking technologies to link up GPU clusters
  • collective algorithms supporting distributed GPU training

Other signals

  • Optimizing ML model performance for training and inference
  • Low-level systems programming and optimization
  • Large-scale training, low-latency inference, high-throughput inference
  • Whole-systems approach including storage, networking, host and GPU level considerations
  • Debugging and optimization using specialized tools