Cx Automation Performance Analyst

Betterment Betterment · Fintech · United States · Customer Experience

This role focuses on the post-launch performance and optimization of AI programs within a customer experience context at a fintech company. The analyst will manage AI tooling, ensure performance, identify degradation, handle incident response, maintain documentation, and build feedback loops to inform future development. Experience in regulated industries is required.

What you'd actually do

  1. Build and maintain dashboards tracking performance across all live AI programs — automation rate, containment, AHT, CSAT, and model quality metrics — using tools like Zendesk Explore, Looker, or similar
  2. Identify performance degradation, intent gaps, hallucination, model drift, and guardrail failures across Zendesk AI, chatbots, ticketbots, and automation workflows
  3. Own the incident response process for AI quality issues — when performance degrades or a guardrail fails, you triage, escalate, and drive resolution with a clear post-mortem
  4. Serve as the primary admin for net-new AI tooling post-launch — including VoiceAI platforms, Zendesk AI agents, and any other tools handed off from the implementation phase
  5. Maintain active optimization backlogs for launched programs, ensuring improvements don't stall after initial implementation

Skills

Required

  • Hands-on experience analyzing performance data from AI or automation tools in a support environment
  • Understanding of common AI failure modes — hallucination, model drift, guardrail gaps, and intent misclassification — and the ability to design QA processes that catch them before they reach customers
  • Proficiency with data visualization and reporting tools (e.g., Zendesk Explore, Looker, Tableau, or similar) and comfortable writing queries to validate AI performance data
  • Strong documentation skills — you write clearly, organize well, and build resources people actually use
  • Analytical and detail-oriented: comfortable building dashboards, running QA analysis, and spotting patterns in messy data
  • Experience with Zendesk (or comparable CX platforms) and AI tools such as Zendesk AI, Decagon, or similar
  • Experience in fintech, financial services, or another regulated industry where AI outputs are subject to compliance review and audit
  • A continuous improvement mindset and systems thinker — you see how knowledge structure, prompt design, monitoring, and feedback loops connect, and you're never satisfied with "it launched"

What the JD emphasized

  • Experience in fintech, financial services, or another regulated industry where AI outputs are subject to compliance review and audit

Other signals

  • post-launch performance management
  • AI program optimization
  • documentation and knowledge management
  • incident response for AI quality issues
  • compliance and audit readiness