Machine Learning Engineer - Model Inference

Apple Apple · Big Tech · Cupertino, CA +1 · Software and Services

Machine Learning Engineer focused on designing, building, and operating services for deploying and serving ML models at scale, with an emphasis on optimizing inference performance (latency, throughput, hardware utilization).

What you'd actually do

  1. Design, implement, test, and maintain scalable machine learning inference services.
  2. Improve inference latency, throughput, availability, and infrastructure efficiency.
  3. Develop benchmarking and profiling tools to identify performance bottlenecks.
  4. Develop techniques such as dynamic batching, caching, quantization, pruning, model compilation, and parallel execution.
  5. Work with machine learning frameworks, inference runtimes, GPUs, and other hardware accelerators.

Skills

Required

  • Python
  • C++
  • Rust
  • Go
  • Data structures
  • Algorithms
  • Operating systems
  • Computer architecture
  • Machine learning fundamentals
  • Deep learning frameworks (PyTorch, TensorFlow, JAX)

Nice to have

  • Model serving technologies (Triton, TensorRT, ONNX Runtime, vLLM, TensorFlow Serving, TorchServe)
  • Inference optimization techniques (quantization, pruning, knowledge distillation, speculative decoding, kernel fusion, continuous batching)
  • GPUs
  • Accelerators
  • Distributed systems
  • Networking
  • High-performance computing
  • Containers
  • Kubernetes
  • Cloud infrastructure
  • Production observability tools
  • Benchmarking LLMs/vision models
  • Model-hardware interaction

What the JD emphasized

  • Improve inference latency, throughput, availability, and infrastructure efficiency.

Other signals

  • Deploying and serving machine learning models at scale
  • Improving inference latency, throughput, and efficiency
  • Optimizing hardware utilization for ML inference