Senior Research Technologist

Zillow Zillow · Consumer · United States · Remote

The role focuses on building and scaling infrastructure for testing agentic AI experiences with real users before they reach production. It involves treating testing infrastructure as a product, defining standards for pre-production testing, and partnering across teams to translate testing needs into practical systems and workflows. The goal is to accelerate product development by improving the speed and repeatability of pre-production AI testing.

What you'd actually do

  1. Build, improve, and scale the infrastructure teams use to test emerging AI-powered experiences with real users before launch.
  2. Treat testing infrastructure like a product by prototyping quickly, learning from real use, and iterating based on what works.
  3. Define clear standards for when and how pre-production testing should happen, what evidence should inform decisions, and how those practices can scale across teams. Build the processes and systems that make these standards repeatable.
  4. Partner with Product, Engineering, Design, Design Technology, and Research to translate testing needs into practical systems and workflows.
  5. Bring technical perspective into research conversations and advocate for strong learning practices in cross-functional decision-making.

Skills

Required

  • technical fluency to partner effectively with engineers and understand how AI-enabled systems are built, tested, and improved
  • strong research judgment and understand what makes user testing and experimentation credible, useful, and actionable
  • collaborate well across Product, Engineering, Design, Research, and hybrid disciplines
  • comfortable with ambiguity and can help define a path forward

Nice to have

  • Scripting or lightweight coding - you've written Python, JavaScript, or similar to automate something or solve a problem, even if you don't write production code regularly
  • API fluency - you've called, configured, or debugged an API directly, and you understand how services connect and exchange data
  • Dev environment experience - comfortable in a terminal, familiar with Git, and understand the difference between staging and production environments
  • Prototyping or configuration tools - experience with tools like Replit, Cursor, Claude Code, or similar environments where you've built or configured something end to end
  • Research or testing platform experience - you've worked with tools used to run studies, configure test environments, or manage participant experiences

What the JD emphasized

  • agentic AI experiences
  • pre-production testing

Other signals

  • agentic AI experiences
  • testing infrastructure
  • pre-production testing process
  • build systems that raise the bar for how Zillow learns before it ships