Software Engineer - Hosted Model Infrastructure

Palantir Palantir · Enterprise · New York, NY · Dev

Software Engineer focused on the infrastructure for deploying and running ML models in production environments, particularly those with constraints like air-gapped networks, limited GPU resources, and strict data sovereignty. The role involves end-to-end ownership of services, including inference engines, GPU scheduling, deployment pipelines, and observability, aiming for continuous delivery of new models and capabilities.

What you'd actually do

  1. deploy AI models to run in variety of environments: air-gapped government networks, forward-deployed defense environments, edge nodes, and enterprises with strict data sovereignty requirements
  2. own services end-to-end, and work across the full stack, from inference engines, GPU scheduling to deployment pipelines, observability, and integration with Palantir's platform
  3. deliver new models and capabilities quickly and continuously

Skills

Required

  • ML model deployment
  • production environments
  • inference engines
  • GPU scheduling
  • deployment pipelines
  • observability
  • full stack development

Nice to have

  • AI infrastructure
  • MLOps

What the JD emphasized

  • constrained GPU resources
  • limited direct access
  • strict data sovereignty requirements

Other signals

  • deploying ML models in production
  • running AI models in air-gapped environments
  • constrained GPU resources
  • full stack ownership from inference engines to deployment pipelines