Sr. AI Engineer

Pendo Pendo · Enterprise · New York, NY · Engineering - Emerging

The Sr. AI Engineer will design and ship applied AI systems, including RAG pipelines, agentic workflows, and LLM-powered features, from prototype through production. This role involves technical decision-making, rigorous model evaluation, MLOps ownership, and full-stack product shipping within a new AI-native product team.

What you'd actually do

  1. Design and build AI systems including RAG pipelines, agentic workflows, and LLM-powered features. You will take work from prototype through production and ensure it can hold up in real customer environments.
  2. Make principled decisions on when to prompt, when to fine-tune, and when to use a different tool entirely. You will explain these tradeoffs clearly so the team can move quickly without sacrificing quality.
  3. Instrument and evaluate model outputs rigorously by defining evaluation frameworks and catching hallucinations early. You will implement guardrails that can withstand real-world load and production use.
  4. Own model deployment, monitoring, latency optimization, cost management, and reliability at scale. You will help ensure AI systems are observable, efficient, and dependable in production.
  5. Contribute across the stack when needed because this team ships products, not just models. You will work across backend and frontend to get AI-powered experiences in front of users.

Skills

Required

  • LLM-powered systems
  • retrieval-augmented generation
  • tool use
  • agent orchestration frameworks
  • system design
  • architecture
  • failure modes
  • tradeoffs
  • technical quality
  • model evaluation
  • evaluation suites
  • overfitting
  • bias-variance tradeoffs
  • hallucinations mitigation
  • model architectures
  • transformers
  • diffusion models
  • production MLOps
  • model deployment
  • monitoring pipelines
  • latency optimization
  • cost optimization
  • reliability optimization
  • full-stack fundamentals
  • backend systems
  • frontend systems
  • user-facing AI products
  • communication skills
  • product thinking

Nice to have

  • fine-tuning foundation models
  • AI safety considerations
  • guardrail frameworks
  • responsible deployment practices
  • agentic or multi-step reasoning systems
  • LangChain
  • LlamaIndex
  • SaaS
  • product analytics
  • net-new team or product area launch

What the JD emphasized

  • Deep hands-on experience building and shipping LLM-powered systems, including retrieval-augmented generation, tool use, and agent orchestration frameworks.
  • Strong technical depth in system design, including choosing the right architecture, identifying failure modes early, and making tradeoffs that hold up across the product lifecycle.
  • Experience owning technical quality beyond your own features, including setting standards, catching problems in review, and improving shared infrastructure and tooling.
  • Strong command of model evaluation, including designing evaluation suites, reasoning about overfitting and bias-variance tradeoffs, and systematically detecting and mitigating hallucinations.
  • Production MLOps experience, including model deployment, monitoring pipelines, and latency, cost, and reliability optimization in a live environment.
  • Strong full-stack fundamentals with the ability to work across backend and frontend systems to ship complete, user-facing AI products.
  • Exceptional communication skills with the ability to explain complex technical decisions clearly to engineers, product managers, and executives.
  • Demonstrated product thinking, including the ability to ask whether something should be built before deciding how to build it.

Other signals

  • design and ship applied AI systems
  • RAG pipelines
  • agentic workflows
  • LLM-powered features
  • prototype through production
  • model evaluation
  • guardrails
  • MLOps ownership
  • model deployment
  • monitoring
  • latency optimization
  • cost management
  • reliability at scale
  • full-stack product shipping
  • AI-native experiences