Senior Quality Engineer

OutSystems OutSystems · Enterprise · San Francisco Bay Area, United States

Senior Quality Engineer focused on defining and implementing quality strategies for AI-powered products. This role involves leading AI-augmented automation, advanced test design for AI features (e.g., prompt injection, hallucination checks), and establishing intelligent CI/CD quality gates. The goal is to ensure AI products are functional, reliable, ethical, and scalable within an enterprise context, leveraging AI tools to accelerate the SDLC.

What you'd actually do

  1. Define and implement the end-to-end test strategy for major AI product areas.
  2. Provide technical leadership for complex automation challenges. You will support the adoption of Cursor and Claude Code to refactor frameworks, generate synthetic data, and automate the creation of sophisticated test suites.
  3. Lead the team in designing coverage for AI-based features, including prompt injection testing, model hallucination checks, and performance benchmarking. You will mentor junior engineers on using Gemini to generate use cases.
  4. Lead triage for all defects. You will analyze trends in model behavior and system performance to implement systemic improvements to the development pipeline.
  5. Define and enforce "smart" quality gates within the CI/CD pipeline. You will hold responsibility for automated suites, ensuring they are reliable, efficient, and capable of catching regressions in both code and model output.

Skills

Required

  • Advanced experience using AI coding assistants (Claude Code, Cursor) and LLMs (Gemini, GPT-4) to accelerate the SDLC.
  • Proven track record of managing product delivery in an Agile environment with a focus on CI/CD and automated quality gates.
  • Proficiency in C#, Python or JavaScript and experience with modern testing frameworks (e.g., Playwright, PyTest).
  • Ability to drive initiatives from ideation to completion with minimal oversight, requiring only strategic alignment with leadership.
  • 5+ years in Quality Engineering or Software Development, with a significant focus on automated systems and AI-based applications.

Nice to have

  • defining how we validate non-deterministic AI outputs and complex microservices
  • guide the quality strategy for high-impact initiatives, ensuring that our AI products are functional, reliable, ethical, and scalable
  • mentor junior engineers on using Gemini to generate use cases
  • analyze trends in model behavior and system performance to implement systemic improvements to the development pipeline
  • document AI-testing standards and formally mentor the team on integrating Generative AI into their daily engineering workflows
  • solve systemic quality issues, balancing the mitigation of technical debt with the rapid pace of AI innovation
  • build the blueprint for how AI-based products are delivered to millions of users

What the JD emphasized

  • non-deterministic AI outputs
  • AI-augmented delivery
  • AI products
  • AI-based features
  • model behavior
  • model output
  • AI Toolchain Expertise
  • AI-based applications
  • AI tools
  • AI-based products
  • agentic systems
  • AI portfolio
  • generative AI
  • AI solutions
  • human-AI collaboration

Other signals

  • defining how to validate non-deterministic AI outputs
  • integrating AI tools to evolve SDLC
  • defining and enforcing 'smart' quality gates within the CI/CD pipeline
  • solving systemic quality issues, balancing mitigation of technical debt with rapid AI innovation