Director of Engineering, AI

Klaviyo Klaviyo · Enterprise · Boston, MA +1 · Engineering

Director of Engineering, AI at Klaviyo, leading teams to implement AI strategy into customer-facing products. Focuses on owning the AI engineering charter, designing the AI platform (LLM-powered and agentic systems), shipping AI-native product experiences, ensuring safety and reliability, and integrating with data/platform stacks. Requires strong leadership, hands-on experience with modern AI/agentic systems, and a product-minded approach to drive AI adoption and impact.

What you'd actually do

  1. Own the AI engineering charter. Define the technical vision and roadmap for AI-powered capabilities on Klaviyo’s platform, and translate it into clear plans, OKRs, and milestones for your teams.
  2. Design and evolve the AI platform. Lead the architecture for LLM-powered and agentic systems — orchestration, tool/function calling, retrieval and memory, evaluation harnesses, observability, and safety guardrails tuned for real-world product workflows.
  3. Ship remarkable, AI-native product experiences. Partner with Product, Design, and partner engineering teams to prioritize the highest-value use cases, break them into shippable iterations, and launch experiences that feel intuitive and powerful — not just demos bolted onto existing UI.
  4. Make safety, reliability, and measurability non-negotiable. Define success metrics and guardrails for AI systems (e.g., time-to-first-value, uplift over baselines, override rates, hallucination/error rates, latency, SLOs), and build the tooling to monitor and continuously improve them in production.
  5. Integrate deeply with Klaviyo’s data and platform stack. Collaborate with Data Platform, Production Infrastructure, and product engineering teams to ensure AI systems can safely and efficiently access the right data while respecting privacy, security, and governance constraints.

Skills

Required

  • Leadership experience (5+ years leading multiple teams and managers)
  • Experience building user-facing products in high-growth SaaS or similar environments
  • Hands-on experience with LLMs, retrieval-augmented generation, or agent frameworks
  • Experience designing multi-step flows, tool calling, memory, evaluation, and safety guardrails
  • Systems and product thinking
  • Outcome-oriented approach with clear success metrics
  • Experience building and developing diverse engineering teams
  • Exceptional collaboration and communication skills
  • Ability to translate complex AI/ML topics for non-technical audiences
  • Adaptability and resilience
  • Experience leading teams in AI-first ways of working

Nice to have

  • Coding every day (implied by 'even if you're not coding every day')
  • Experience with prompt and chain design
  • Experience with human-in-the-loop review
  • Experience with rigorous experimentation, A/B testing, and fast feedback loops
  • Experience with AI tools in day-to-day engineering development cycles

What the JD emphasized

  • AI strategy into real, daily leverage for hundreds of thousands of customers
  • AI-powered capabilities across Klaviyo’s product surface
  • product experiences embedded throughout Klaviyo
  • technical vision and execution strategy for AI-powered capabilities
  • partnering deeply with Product, Design, ML, Data Science, and GTM
  • define what to build, ship it quickly and safely, and measure real impact in customer outcomes — not just model demos
  • hands-on, execution-first role for a product-minded engineering leader
  • fluent in modern LLMs, agent patterns
  • bring AI capabilities to production at scale
  • Own the AI engineering charter
  • Design and evolve the AI platform
  • LLM-powered and agentic systems
  • orchestration, tool/function calling, retrieval and memory, evaluation harnesses, observability, and safety guardrails
  • real-world product workflows
  • Ship remarkable, AI-native product experiences
  • prioritize the highest-value use cases
  • break them into shippable iterations
  • launch experiences that feel intuitive and powerful — not just demos bolted onto existing UI
  • Make safety, reliability, and measurability non-negotiable
  • Define success metrics and guardrails for AI systems
  • time-to-first-value, uplift over baselines, override rates, hallucination/error rates, latency, SLOs
  • build the tooling to monitor and continuously improve them in production
  • Integrate deeply with Klaviyo’s data and platform stack
  • safely and efficiently access the right data while respecting privacy, security, and governance constraints
  • Raise the bar on AI engineering craft
  • Build shared patterns, libraries, and best practices for prompt and chain design, evaluation, human-in-the-loop review, and observability
  • other teams can reuse
  • champion rigorous experimentation, A/B testing, and fast feedback loops
  • Lead high-performing engineering teams (ICs and managers)
  • Hire, develop, and retain high-performing, inclusive teams
  • set crisp expectations, coach for impact
  • create a culture of urgency, accountability, and psychological safety
  • Partner across Klaviyo to land changes
  • roll out new AI capabilities safely
  • enable the field
  • close the loop between customer feedback, product decisions, and system improvements
  • Measure what matters
  • Own a clear scorecard for your area
  • adoption, activation, uplift vs. baselines, quality/satisfaction, AI system reliability, and developer velocity
  • use it to drive prioritization and investment decisions
  • Transform workflows by putting AI at the center
  • building smarter systems and ways of working from the ground up
  • continuously experimenting with AI tools
  • testing, learning, and sharing insights to keep Klaviyo ahead of the curve
  • An experienced, product-minded engineering leader
  • ~10+ years in software engineering, including 5+ years leading multiple teams and managers building user-facing products in high-growth SaaS or similar environments
  • You’re energized by building things customers love, not just infrastructure for its own sake
  • Hands-on with modern AI and agentic systems
  • You’ve led teams building with LLMs, retrieval-augmented generation, or agent frameworks
  • designing multi-step flows, tool calling, memory, evaluation, and safety guardrails
  • even if you’re not coding every day
  • A deep systems and product thinker
  • You can dive into architecture and data flows, but also reason from first principles about customer workflows, jobs-to-be-done, and where AI can produce step-change improvements vs. incremental shortcuts
  • An outcome-oriented operator
  • You define clear success metrics and hold teams accountable to them
  • you’re comfortable saying “no,” narrowing scope, and iterating fast
  • A builder of high-performing, inclusive teams
  • You’ve hired and developed diverse engineering teams, grown new leaders, and created cultures where people do the best work of their careers while feeling respected and included
  • An exceptional collaborator and communicator
  • You translate complex AI/ML and systems topics into clear narratives for non-technical partners
  • you influence peers and executives, drive alignment across Product, Engineering, Data, and GTM
  • model Klaviyo’s values in how you work
  • Adaptable and resilient
  • You stay close to the work, dive deep when needed, and lead your teams through ambiguity and change with clarity and calm
  • A practitioner of AI-first ways of working
  • You know how to lead your whole team to use AI in day-to-day engineering development cycles
  • you’ve experimented with or practiced fully agen

Other signals

  • AI strategy into real, daily leverage for hundreds of thousands of customers
  • AI-powered capabilities across Klaviyo’s product surface
  • modern LLMs, agent patterns
  • bring AI capabilities to production at scale