Principal AI Engineer

Salesforce Salesforce · Enterprise · Mexico City, Mexico

Seeking a Principal AI Engineer to build and maintain the ML/AI platform powering autonomous AI agents at enterprise scale. This role focuses on agent harness infrastructure, sandboxed execution, evaluation frameworks, CI/CD pipelines for agent deployment, and overall platform architecture and reliability. The goal is to enable the development, testing, and trusted production deployment of AI agents across various Salesforce verticals.

What you'd actually do

  1. Design and build agent harness infrastructure: the scaffolding that wraps LLM calls, manages tool use, handles retries, enforces policy, and feeds results back into iterative improvement loops.
  2. Implement agentic loop patterns with multi-turn reasoning, tool orchestration, memory management, and structured output handling as reusable platform primitives
  3. Build the agent flywheel: automated pipelines that collect agent traces, surface regressions, route failures to evaluation, and close the loop from production signal back to prompt/model improvement
  4. Own the end-to-end lifecycle from agent experiment to production deployment, including versioning, rollout controls, and rollback mechanisms
  5. Build sandboxed execution environments for agent tools with isolating code execution, API calls, and file system access so agents can act without unconstrained blast radius

Skills

Required

  • Platform Engineering
  • ML Infrastructure Engineering
  • agent systems design
  • evaluation tooling
  • LLM calls
  • tool use
  • multi-turn reasoning
  • tool orchestration
  • memory management
  • structured output handling
  • automated pipelines
  • production deployment
  • versioning
  • rollout controls
  • rollback mechanisms
  • sandboxed execution environments
  • code execution isolation
  • API call isolation
  • file system access isolation
  • tiered autonomy models
  • replay capabilities
  • dry-run capabilities
  • evaluation frameworks
  • vendor tools
  • open source tools
  • in-house built tools
  • task success evaluation
  • tool selection accuracy evaluation
  • trajectory evaluation
  • hallucination rates evaluation
  • latency evaluation
  • cost evaluation
  • eval datasets
  • golden trace libraries
  • regression test suites
  • agent traces instrumentation
  • LLM calls instrumentation
  • tool invocations instrumentation
  • intermediate reasoning instrumentation
  • final outputs instrumentation
  • Grafana
  • observability tooling
  • agent quality metrics tracking
  • continuous quality improvement
  • latency improvement
  • cost improvement
  • production traces analysis
  • prompt tuning
  • tool calling optimizations
  • context engineering
  • model selection optimization
  • distillation
  • fine-tuning
  • SFT
  • DPO
  • RLHF
  • curated trace data
  • A/B tests
  • shadow deployments
  • replay against golden traces
  • CI/CD pipelines
  • GitHub Actions
  • ArgoCD
  • code deployment
  • evaluation gates
  • Docker builds
  • package builds
  • security scanning
  • agent integration tests
  • self-healing CI patterns
  • developer self-service interfaces
  • ML engineers
  • data scientists
  • platform components architecture
  • infrastructure architecture
  • agent harnesses architecture
  • evaluation pipelines architecture
  • observability architecture
  • architecture diagrams
  • platform vision
  • alerting
  • PagerDuty
  • platform health monitoring
  • agent-specific signals monitoring
  • error rates monitoring
  • tool call failures monitoring
  • eval score drift monitoring
  • security best practices
  • auditable traces
  • access controls
  • security reviews
  • compliance for agent workloads

What the JD emphasized

  • agent harness infrastructure
  • agent flywheel
  • agent experiment to production deployment
  • sandboxed execution environments
  • agent evaluation
  • agent behavior
  • evaluation frameworks
  • agent traces
  • agent code change
  • agent quality metrics
  • agent design
  • agent optimization
  • agent evaluation gates
  • agent integration tests
  • agent-based automation
  • agent tools
  • agent infrastructure
  • agent-specific signals
  • agent infrastructure adheres to security best practices
  • agent workloads

Other signals

  • building the next generation of our ML/AI platform
  • powers autonomous AI agents at enterprise scale
  • intersection of platform infrastructure and agent systems engineering
  • designing the harnesses, sandboxes, and evaluation frameworks that let AI agents be developed, tested, and trusted in production
  • own the end-to-end lifecycle from agent experiment to production deployment
  • implement evaluation frameworks for agent behavior
  • instrument agent traces end-to-end
  • drive continuous quality, latency, and cost improvements across deployed agents
  • build and optimize CI/CD pipelines ... that cover not just code deployment but agent evaluation gates
  • create architecture diagrams and drive long-term platform vision