Software Engineer Ii- Product Recommendations

Klaviyo · Enterprise · Boston, MA · Engineering

Senior Software Engineer focused on building and operating machine learning-powered backend systems for product recommendations. This role involves designing and implementing data pipelines, ranking models, and inference systems, with a strong emphasis on backend services, distributed systems, and integrating ML into production. The engineer will also drive the development of AI systems like vector search and agentic use cases, and contribute to observability and A/B testing.

What you'd actually do

  1. Lead the design, architecture, and operation of backend services that power product recommendations across Klaviyo experiences (email, SMS, KAgent, onsite, etc.), upholding standards for reliability, performance, and clear APIs.
  2. Architect and maintain robust, large-scale data processing pipelines (e.g., using Apache Spark or similar frameworks) that transform raw events and catalog data into high-quality features and inputs for recommendation models, ensuring data quality and lineage.
  3. Collaborate closely with ML engineers and product stakeholders to strategically productionize recommendation models—defining high-level interfaces, robust feature contracts, and advanced deployment patterns for batch and/or real-time inference systems.
  4. Drive the development of ML/AI systems such as vector search that power recommendation, semantic search, and sophisticated agentic use cases.
  5. Implement and evolve data and service observability (metrics, logging, tracing, dashboards) to proactively ensure recommendations are correct, fast, and highly available for all customers.

Skills

Required

  • 5+ years of software engineering experience
  • building and operating mission-critical backend services and systems in a production environment
  • backend and distributed systems at scale
  • high-throughput, highly available services
  • optimizing for latency, reliability, and operability
  • Python
  • cloud-native architectures (AWS preferred)
  • container orchestration (e.g., Kubernetes)
  • managing infrastructure and CI/CD pipelines
  • data-driven decision making and A/B testing
  • designing and querying data models in relational, analytical, and NoSQL datastores (e.g., Postgres, MySQL, data warehouses, Redis, vector databases)
  • modern DevOps practices (CI/CD, monitoring, alerting)
  • owning multi-component projects end-to-end
  • technical collaborator and communicator
  • actively experimented with AI in work or personal projects

Nice to have

  • Previous experience working on product recommendation systems or adjacent ML-powered features (ranking, personalization, search, or similar)
  • Experience with big data frameworks such as Apache Spark (or similar technologies like Flink, Beam, etc.) for architecting and building complex batch or streaming pipelines
  • Experience in AI/ML systems and products, such as integrating models into production systems, building features powered by ML, or contributing to the ML infrastructure
  • Experience training and iterating on machine learning models (e.g., for ranking, personalization, search, or similar)

What the JD emphasized

  • backend services
  • data processing pipelines
  • recommendation models
  • inference systems
  • vector search
  • agentic use cases
  • observability
  • backend and distributed systems at scale
  • high-throughput, highly available services
  • optimizing for latency, reliability, and operability
  • cloud-native architectures
  • container orchestration
  • DevOps practices
  • large-scale data and recommendation systems
  • multi-component projects end-to-end
  • AI/ML systems and products

Other signals

  • productionize recommendation models
  • vector search that power recommendation
  • semantic search
  • sophisticated agentic use cases
  • ML/AI systems and products