Staff Advanced Analytics, Digital & AI Products

Airbnb Airbnb · Consumer · Bangalore, India · Analytics

Staff Advanced Analyst role focused on building and scaling analytical solutions for AI products within the Customer Service organization. This role will partner with product and business leaders to provide insights, drive product analytics, and lead measurement for AI systems, including AI Assistants and agent-assisting tools. Key responsibilities include owning the product analytics roadmap, collaborating with engineering on logging and observability for agentic and LLM workflows, designing and evaluating A/B tests for AI features, and translating insights into strategic recommendations. The role requires expertise in Python, SQL, A/B testing, NLP/ML/DL techniques, LLM architectures, and familiarity with Agentic AI systems and AI observability.

What you'd actually do

  1. Owning the product analytics roadmap, prioritization & delivery of solutions in the Contact Center Product space.
  2. Own projects from start to end: building out timelines, key milestones, providing regular updates to product managers, analytics management and delivery against agreed timelines.
  3. Lead end-to-end measurement for AI systems, aligning metrics with business outcomes, user experience, and trustworthiness
  4. Collaborate with engineering to define scalable logging and ensure observability across agentic and LLM workflows.
  5. Lead the design and evaluate A/B and causal tests to quantify impact of AI features and optimizations.

Skills

Required

  • Python
  • SQL
  • A/B testing platforms and best practices
  • EDA
  • hypothesis testing
  • significance testing
  • regression
  • clustering techniques
  • NLP/Text Mining
  • machine learning
  • deep learning techniques
  • language model fine-tuning
  • LLM architectures (e.g., prompt chains, retrieval augmentation, orchestration frameworks like LangChain or DSPy)
  • Agentic AI systems
  • human-in-the-loop design
  • AI observability best practices
  • designing and building metrics
  • building prototypes with data pipelines

Nice to have

  • vector stores
  • embeddings
  • prompt instrumentation
  • structured logging of LLM interactions
  • Contact center domain
  • market place domain knowledge

What the JD emphasized

  • AI Assistant
  • agent assisting tools
  • AI systems
  • agentic and LLM workflows
  • AI features
  • AI analytics
  • LLM architectures
  • Agentic AI systems
  • AI observability

Other signals

  • AI Assistant
  • agent assisting tools
  • AI systems
  • agentic and LLM workflows
  • AI features
  • AI analytics
  • LLM architectures
  • Agentic AI systems
  • AI observability