Senior Software Quality Engineer

Adobe Adobe · Enterprise · San Francisco, CA

Senior Software Quality Engineer for a new AI-first web application, focusing on end-to-end quality strategy, test automation, and establishing evaluation harnesses for AI features. The role involves ensuring quality across the prompt/agent layer, user experience, and generated output, embedding quality into CI/CD, and collaborating with engineering and product teams.

What you'd actually do

  1. Own the end-to-end quality strategy for a new AI-first web application, spanning the prompt and agent layer, the in-browser experience, and the generated output.
  2. Compose, build, and maintain modern web test automation (Playwright / Cypress, TypeScript/JavaScript) and the underlying infrastructure that keeps it fast, deterministic, and developer-friendly.
  3. Stand up evaluation harnesses for automated workflows: golden prompt suites, regression baselines, model/version comparisons, hallucination and safety checks, and human-in-the-loop review pipelines.
  4. Ensure comprehensive quality for a real-time web product — performance, resilience, accessibility (WCAG), and cross-browser/cross-device coverage.
  5. Embed quality into CI/CD: build release gates that catch real regressions, reduce flake, and turn production telemetry into pre-submit signal.

Skills

Required

  • 8+ years of experience in customer -facing software quality / SDET positions
  • Automation experience with modern web frameworks (Playwright, Cypress, or similar)
  • strong coding skills in TypeScript/JavaScript
  • Experience designing test strategy for AI / ML / LLM-powered features
  • Strong fundamentals in API testing
  • performance and load testing
  • observability tooling (Datadog, Splunk, Sentry, OpenTelemetry, or similar)
  • Excellent collaboration and communication
  • able to influence engineers and PMs early in the build cycle

Nice to have

  • Python
  • Prior experience with creative tools, design-to-code, browser-based graphics (Canvas/WebGL), or real-time collaboration stacks.

What the JD emphasized

  • AI-first web application
  • prompt and agent layer
  • evaluation harnesses
  • AI / ML / LLM-powered features
  • prompt regression
  • evaluation datasets
  • model drift
  • hallucination and safety checks
  • human-in-the-loop review pipelines

Other signals

  • AI-first web application
  • prompt and agent layer
  • evaluation harnesses
  • AI / ML / LLM-powered features