Senior Software Engineer, AI - Simulation

Brex Brex · Fintech · San Francisco, CA +1 · Engineering

This role focuses on building and owning a simulation and validation platform for AI-powered products at Brex. The primary goal is to ensure AI features are continuously tested under realistic conditions, catching regressions and validating new capabilities before they reach customers. The platform will use synthetic data and scenario generation to create a self-improvement loop, with the engineer having the authority to block launches if quality standards are not met. This is an engineering role focused on quality assurance and robustness of AI systems.

What you'd actually do

  1. Evolve and scale a simulation and validation platform to continuously stress-test Brex’s AI products using large-scale synthetic data.
  2. Develop evaluation mechanisms to detect regressions, edge cases, and failure modes in non-deterministic systems prior to customer impact.
  3. Run continuous testing in isolated, production-like environments and set quality gates, including authority to block launches when standards are not met.
  4. Partner closely with product engineering teams to improve system testability, isolation, and overall robustness.
  5. Define and uphold company-wide standards for AI feature quality, using simulation-driven insights to guide prompt and system evolution, while driving the technical bar for a small, high-impact team.

Skills

Required

  • Experience building and owning complex software systems where correctness, reliability, and failure modes are critical.
  • Experience designing test, simulation, or validation systems for non-deterministic or high-stakes environments.
  • Strong systems thinking, with the ability to decompose complex problems and reduce incidental complexity.
  • Demonstrated ability to influence product architecture through quality, reliability, and launch-readiness constraints.
  • Comfort operating with real authority and cross-team influence, including blocking launches and partnering across seniority levels to raise the quality bar.

Nice to have

  • Previous experience on simulation or testing of inherently complex systems (AI/ML, distributed systems, robotics, IoT, medical devices, aerospace, defense, etc.).
  • Experience building internal platforms that became critical shared infrastructure.
  • Strong intuition for failure modes in real-world systems.

What the JD emphasized

  • ensure that every AI surface at Brex is continuously scrutinized under extreme, realistic conditions
  • catch regressions before customers do
  • validate new capabilities safely
  • evolve prompts and agent behavior continuously
  • exercising real products, breaking them creatively
  • forcing the organization to get better
  • detect regressions, edge cases, and failure modes
  • set quality gates, including authority to block launches
  • define and uphold company-wide standards for AI feature quality
  • driving the technical bar for a small, high-impact team

Other signals

  • simulation and validation platform
  • synthetic data and scenario generation
  • catch regressions before customers do
  • validate new capabilities safely
  • evolve prompts and agent behavior continuously
  • run against isolated, ephemeral environments
  • exercising real products, breaking them creatively
  • forcing the organization to get better
  • detect regressions, edge cases, and failure modes
  • set quality gates, including authority to block launches
  • define and uphold company-wide standards for AI feature quality
  • driving the technical bar for a small, high-impact team