Staff Software Engineer - Microservices

ServiceNow ServiceNow · Enterprise · Hyderabad, India · Engineering, Infrastructure and Operations

Staff Software Engineer for the Agentic AI team at ServiceNow, focusing on building the foundational infrastructure for AI Agents. The role involves designing and developing agentic pipelines using Python and Java, covering agent reasoning, memory, orchestration, and evaluation at enterprise scale. It also includes contributing to research initiatives in agent learning and evaluation, and ensuring reliability and performance for LLMs and Agentic AI. Responsibilities include leading AI solution design, end-to-end delivery, prompt engineering, pipeline optimization, technical leadership, and championing observability and evaluation for AI systems.

What you'd actually do

  1. Lead the design and architecture of AI solutions , shaping how reasoning, memory, and orchestration come together in production systems.
  2. Own end-to-end delivery—from design through deployment—ensuring solutions are scalable, reliable, and performant.
  3. Work hands-on with modern AI frameworks and open-source SDKs, with a strong emphasis on prompt engineering and pipeline optimization.
  4. Stay current with emerging advancements in LLMs and Agentic AI, integrating relevant techniques into our platform roadmap.
  5. Provide technical leadership by mentoring engineers, conducting code reviews, and defining best practices that foster a culture of innovation and quality.

Skills

Required

  • Experience in leveraging and critically evaluating AI integration into work processes, decision-making, and problem-solving.
  • 9+ years of experience designing and implementing complex enterprise applications, with proven success in conversational AI and NLP platforms.
  • Strong development skills in Java and Python, including building production-grade AI/ML systems.
  • Hands-on expertise in LLM-powered applications and tooling, including frameworks such as LangChain and LangGraph, and vendor APIs such as OpenAI and Anthropic.
  • Deep understanding of agentic reasoning pipelines, evaluation frameworks, and advanced NLP techniques.
  • Strong problem-solving abilities with a bias for action, a customer-first mindset, and the ability to operate effectively in fast-paced environments.
  • Demonstrated analytical skills in evaluating model performance, optimizing inference latency and cost, and improving system efficiency end-to-end.
  • Proven ability to translate complex AI concepts into clear business value and communicate effectively with both technical and non-technical stakeholders.
  • Experience providing technical leadership and guiding successful AI/ML engineering projects across teams or domains.

Nice to have

  • Publications, technical blogs, conference talks, or open-source contributions in LLMs, GenAI, or AI systems engineering.

What the JD emphasized

  • production systems
  • production-grade AI/ML systems
  • production workloads

Other signals

  • AI Agents framework
  • agent reasoning, memory, orchestration, and evaluation
  • LLMs and Agentic AI
  • modern AI frameworks and open-source SDKs
  • prompt engineering and pipeline optimization
  • observability and evaluation for AI systems
  • logging, metrics, and automated evaluation frameworks