AI Engineer III

Premera Blue Cross Premera Blue Cross · Insurance · Telecommuter

AI Engineer III role focused on developing and productionizing multi-agent AI systems, knowledge-augmented retrieval (including GraphRAG), and robust data pipelines for complex business problems, leveraging cloud resources and observability.

What you'd actually do

  1. Develop comprehensive multi-agent systems and frameworks for AI applications and products using orchestration patterns like manager/worker and peer collaboration.
  2. Develop prototypes and production-ready AI products to validate AI/ML solutions implementing GraphRAG and knowledge-augmented retrieval.
  3. Develop, tune, and productionize AI systems using cloud resources such as Azure AI Foundry, Kubernetes, custom vector database deployments, and Cognitive Services.
  4. Contribute to development of robust data pipelines for knowledge graph construction, entity extraction, and graph-based retrieval systems.
  5. Develop specifications context governance, prompt versioning, and agent evaluation frameworks to deploy AI models and incorporate them into applications.

Skills

Required

  • Bachelor's Degree in Computer Science, Statistics, Mathematics, or a related field, or 2+ years of experience in a related, professional IT/analytics position.
  • Minimum of 3 years of industry experience in developing, deploying, and maintaining AI or ML systems.
  • Expert proficiency in multi-agent orchestration patterns and workflows.
  • Deep understanding of GraphRAG and knowledge graph augmentation techniques.
  • Ability to implement advanced fusion methods and cross-encoder re-ranking.
  • Proficiency with agent communication protocols (A2A, MCP) at scale.
  • Experience with agent evaluation frameworks and benchmarks (AgentBench, HELM).
  • Advanced prompt engineering including iterative refinement and critique loops.
  • Ability to design and implement production guardrails and safety systems.
  • Knowledge of vector database scaling strategies and optimization.
  • Experience with distributed agent systems and message brokers.
  • Proficiency in implementing agent observability and distributed tracing.
  • Understanding of token optimization and context window management.
  • Experience with continuous indexing pipelines and embedding updates.

Nice to have

  • At least 3 years of experience in developing complex multi-agent systems using frameworks like Microsoft’s Agent Framework, LangChain/LangGraph, or custom orchestration.
  • Previous experience with GraphRAG implementation and knowledge graph constructions from unstructured text.
  • Experience with advanced retrieval techniques including cross-encoder re-ranking and fusion methods beyond RRF.
  • Certifications in agent development frameworks or demonstrable contributions to open-source agent projects.
  • Minimum of 3 years of experience in successfully productionizing AI models, including constructing scalable data pipelines and establishing comprehensive monitoring systems.
  • At least 1 year of experience instrumenting multi-agent AI systems with OpenTelemetry and Semantic Convention standards, including distributed tracing across agent communications, tool calls, and LLM interactions for observability, debugging, and compliance.
  • 3 or more years working within Agile-like teams and environments.

What the JD emphasized

  • Minimum of 3 years of industry experience in developing, deploying, and maintaining AI or ML systems.
  • At least 3 years of experience in developing complex multi-agent systems using frameworks like Microsoft’s Agent Framework, LangChain/LangGraph, or custom orchestration.
  • Minimum of 3 years of experience in successfully productionizing AI models, including constructing scalable data pipelines and establishing comprehensive monitoring systems.
  • At least 1 year of experience instrumenting multi-agent AI systems with OpenTelemetry and Semantic Convention standards, including distributed tracing across agent communications, tool calls, and LLM interactions for observability, debugging, and compliance.

Other signals

  • multi-agent systems
  • knowledge-augmented retrieval
  • GraphRAG
  • production implementation
  • orchestration patterns