Sr. Staff Software Engineer - AI Chat

Eli Lilly Eli Lilly · Pharma · Indianapolis, IN

Senior Staff Software Engineer to own the technical direction and delivery of ChatNow's core platform, which includes the routing engine that matches intent to the right agent, the integration layer that connects a growing ecosystem of AI capabilities, and the experience layer that makes it all feel effortless. The role involves defining the architecture and driving implementation of the conversation routing engine, owning the agent integration layer, building streaming chat interfaces, and integrating LLM capabilities through a model gateway with multi-model routing and fallback.

What you'd actually do

  1. You'll define the architecture and drive implementation of ChatNow's conversation routing engine — the system that takes a user's intent and connects it to the right AI agent, model, or workflow.
  2. You'll own the agent integration layer, designing how ChatNow connects to Lilly's expanding network of specialized AI agents on Cortex.
  3. You'll build streaming chat interfaces in React and TypeScript, backed by scalable services that handle thousands of concurrent sessions with low latency across multiple AI providers.
  4. You'll own features end to end. Design, build, test, deploy, monitor.
  5. You'll integrate LLM capabilities through Cortex's model gateway, supporting multi-model routing with fallback and load balancing.

Skills

Required

  • 10+ years of professional software engineering experience, with demonstrated technical leadership at scale
  • 5+ years in TypeScript/JavaScript, Python and Go (with ability to use AI coding tools proficiently), architecting and building production web applications with React or equivalent
  • 5+ years experience designing RESTful or GraphQL APIs at scale
  • Deep hands-on experience with Kubernetes — not just deploying, but operating, debugging, and tuning
  • Strong CI/CD and GitOps practices
  • Bachelor's degree in Computer Science, Software Engineering, or a related field

Nice to have

  • Experience integrating LLM APIs into production — streaming, function calling, tool use
  • Hands-on work with conversational AI systems: routing, intent classification, context management, or multi-agent orchestration
  • Experience with real-time systems: WebSockets, SSE, event-driven architectures, or workflow orchestration
  • Familiarity with MCP, A2A, or emerging AI interoperability protocols
  • Active use of agentic coding tools in your daily workflow
  • A track record of shipping user-facing products with high reliability and polish
  • Experience setting technical direction across multiple teams or workstreams

What the JD emphasized

  • own the technical direction and delivery
  • ship code that the entire company uses every day
  • own features end to end
  • operate ChatNow on Cortex and Hangar
  • AI-native engineering at staff level
  • defining the framework that the rest of the organization builds on
  • make technical decisions that ripple across teams
  • mentor engineers
  • raise the standard for what production AI software looks like at Lilly

Other signals

  • enterprise AI platform
  • AI agents
  • routing engine
  • integration layer
  • streaming chat interfaces
  • multi-model routing
  • agentic coding tools