Senior Software Engineer, Machine Learning Infrastructure

Handshake Handshake · Enterprise · San Francisco, CA · Engineering

Senior Software Engineer focused on building and operating ML infrastructure and platforms for both Handshake's core business and Handshake AI. The role involves developing scalable systems for data pipelines, feature stores, training, model serving, LLM platforms, evaluation infrastructure, and inference optimization.

What you'd actually do

  1. Build and operate the shared infrastructure behind production ML and AI, including data pipelines, feature stores, training, and model serving.
  2. Develop and scale our LLM platform, including provider integrations, orchestration, observability, and controls for cost, latency, and reliability.
  3. Build evaluation infrastructure, including LLM eval harnesses, benchmarks, and quality measurement pipelines.
  4. Support post-training workflows, including fine-tuning, reinforcement learning pipelines, and supporting data infrastructure.
  5. Optimize inference infrastructure for open and fine-tuned models, including GPU serving, batching, and autoscaling.

Skills

Required

  • Python
  • Go
  • TypeScript
  • cloud infrastructure (AWS, GCP)
  • Kubernetes
  • Docker
  • Terraform
  • CI/CD
  • operating production services
  • ML infrastructure (model serving, training pipelines, feature stores, embeddings, ML observability)
  • modern data platforms (BigQuery, Airflow, Spark, Beam/Dataflow, streaming pipelines)
  • production systems with LLMs or generative AI (orchestration, provider APIs, observability, performance optimization)
  • systems design

Nice to have

  • Ray
  • Anyscale
  • KubeRay
  • Ray Serve
  • vLLM
  • Triton
  • PyTorch
  • GPU-backed inference and training
  • LLM evaluation frameworks
  • benchmarking systems
  • quality regression testing
  • Vertex AI
  • Bigtable
  • Redis
  • feature platform infrastructure
  • fine-tuning
  • RLHF
  • reinforcement learning
  • reward modeling
  • agentic systems
  • MCP integrations
  • tool use
  • memory systems
  • voice AI applications

What the JD emphasized

  • production ML and AI systems
  • LLM platform
  • evaluation infrastructure
  • post-training workflows
  • inference infrastructure
  • production software engineering experience
  • cloud infrastructure
  • Kubernetes, Docker, Terraform, CI/CD
  • ML infrastructure
  • modern data platforms
  • production systems with LLMs or generative AI

Other signals

  • building shared infrastructure for production ML and AI systems
  • developing and scaling LLM platform
  • building evaluation infrastructure
  • supporting post-training workflows
  • optimizing inference infrastructure