Machine Learning Engineer 5

Adobe Adobe · Enterprise · San Jose, CA

Machine Learning Engineer to build and ship LLM-powered systems simulating consumer audiences for pre-testing ads and content. Develops inference and reasoning harnesses on LLMs, fine-tunes LLMs on data, and builds evaluation datasets and benchmarks. Requires substantial hands-on experience with LLM applications in production, including agentic systems, fine-tuning, and RLHF/RLAIF. Experience with MLOps and cloud ML services is needed. The role focuses on bringing cutting-edge AI research in synthetic audiences into a production product.

What you'd actually do

  1. Design, build, and ship LLM-powered systems that simulate consumer audiences end-to-end, from proof-of-concept to production.
  2. Develop complex inference and reasoning harnesses on top of frontier LLMs, agentic flows, persona conditioning, retrieval, and sampling strategies tuned for distributional fidelity.
  3. Fine-tune LLMs on survey, panel, and behavioral data to improve alignment with real-world audience distributions; own the full loop from data curation through eval.
  4. Build the evaluation datasets, benchmarks, and harnesses that define what "good" means for synthetic audience quality - distributional fidelity, behavioral validity, subgroup calibration.
  5. Partner with product management, applied science, and engineering to translate a fast-moving research literature into shipping product features.

Skills

Required

  • Python
  • data structures
  • algorithms
  • PyTorch
  • Hugging Face
  • vLLM
  • W&B
  • MLOps practices and pipelines
  • cloud ML services (AWS, GCP, Azure)

Nice to have

  • synthetic audiences
  • persona simulation
  • LLM-based human behavior modeling
  • public opinion or survey data
  • panel-based consumer research data

What the JD emphasized

  • Substantial hands-on experience building LLM-based applications in production.
  • Demonstrated experience designing and shipping complex inference harnesses on top of large language models (agentic systems, structured reasoning, sampling/decoding strategies, RAG).
  • Proven track record of building evaluation datasets and harnesses — you have opinions about what makes an eval load-bearing versus theater.
  • Shipped a customer-facing Gen AI feature from proof-of-concept to production end-to-end.

Other signals

  • LLM-powered systems
  • synthetic audiences
  • simulated consumers
  • pre-test ads, campaigns, and content