Lead Product Analyst - Monday Agents

Monday.com Monday.com · Enterprise · Tel-Aviv, Israel · Product

Lead Product Analyst for monday Agents, responsible for defining the analytical direction and measurement foundation for their autonomous workforce ecosystem. This role involves building agent evaluation frameworks, analyzing user interactions with autonomous workflows, and ensuring safe and reliable agent execution.

What you'd actually do

  1. Build and own monday Agents' measurement foundation—what we instrument, how we evaluate outcomes, and the success metrics product and leadership use to steer the roadmap across the monday ecosystem.
  2. Design and run rigorous agent evaluation at scale - from "LLM as judge" setups to behavioral profiling - so we can benchmark what works, what fails, and where to invest next.
  3. Build the analytical frameworks needed to monitor agent guardrails, permissions boundary compliance, and security patterns, ensuring autonomous execution remains completely safe and enterprise-ready.
  4. Translate complex execution logs and multi-step agent trajectories into actionable, executive-ready insights that accelerate feature deployment.
  5. Partner with product leadership to turn abstract AI capabilities into structured, data-driven product roadmaps that prioritize user value and system reliability.

Skills

Required

  • SQL
  • Python
  • defining how AI features are judged: outcome quality, reliability, guardrails, drift from intent, and the tradeoffs between cost, latency, and user value
  • exploring agent execution data at scale
  • partnering with data engineering on instrumentation
  • turning multi-step execution data (runs, traces, JSON logs) into metrics, dashboards, and definitions

Nice to have

  • experience ideally on agents, automation, or similarly complex surfaces where user outcomes matter more than funnel volume

What the JD emphasized

  • agent evaluation at scale
  • multi-step execution data
  • agent guardrails
  • permissions boundary compliance
  • security patterns
  • autonomous execution
  • agent accuracy, latency, execution success rates, and token consumption tradeoffs

Other signals

  • building agent evaluation frameworks
  • defining agent success metrics
  • measuring agent guardrails and safety
  • analyzing multi-step agent trajectories
  • optimizing agent accuracy, latency, and cost tradeoffs