Senior Analyst, AI Workflows & Automation

Roblox Roblox · Consumer · Gurgaon, India · Safety Operations (Non-Tech)

This role focuses on designing, prototyping, and scaling AI-augmented workflows for Trust & Safety Operations at Roblox. The Senior Analyst will translate operational pain points into AI workflows, partner with product and engineering, and build evaluation frameworks to ensure automations are safe, reliable, and measurable. The role involves using LLM agents, existing rules engines, and human review to improve efficiency and consistency, with a strong emphasis on analytical skills and cross-functional communication.

What you'd actually do

  1. Design AI‑augmented workflows that combine LLM agents, existing rules engines, and human review to improve speed, quality, and consistency
  2. Prototype and iterate AI solutions using internal and external (accepted) tooling (e.g., Claude and other LLMs, internal agentic/AI workflow platforms, and Builder AI code intelligence) to turn workflow designs into working agents or automations.
  3. Define success metrics and evaluation plans (e.g., automation rate, handle time reduction, error/rollback rate, reviewer satisfaction) and build or integrate with eval pipelines to measure agent performance over time.
  4. Partner with product, engineering, and operations to prioritize use cases, scope requirements, and align AI workflows with product roadmaps and control/compliance expectations.
  5. Continuously improve workflows by monitoring performance, surfacing insights, and iterating on prompts, policies, routing, and UX to keep systems robust as volumes, abuse patterns, and regulations evolve.

Skills

Required

  • 5+ years of experience in technical operations, data science/analytics, or product/engineering roles
  • at least 2+ years working directly with workflow automation, rules engines, or tooling for operations teams
  • Demonstrated experience designing, implementing, or maintaining complex business workflows
  • Strong technical fluency: comfortable reading and writing simple scripts or code (Python or similar), working with APIs, and collaborating closely with engineers to ship workflow changes and integrations.
  • Hands-on experience with LLM-based tools (e.g., Claude) or other AI platforms, including prompt design, evaluation, and iteration to improve quality and reliability.
  • Strong analytical skills, with experience defining metrics, building dashboards or analyses, and using data to make prioritization and design decisions.
  • Excellent cross-functional communication skills

Nice to have

  • Experience in Trust & Safety, compliance, or risk-sensitive workflows
  • Prior work evaluating AI agents or automation quality

What the JD emphasized

  • create workflows
  • AI workflows
  • workflow automation
  • complex business workflows
  • LLM-based tools
  • evaluating AI agents or automation quality

Other signals

  • designing AI-augmented workflows
  • LLM agents
  • prototype and iterate AI solutions
  • build evaluation frameworks