Overview
As a Principal Software Engineer on the Azure Artificial Intelligence Core team at Microsoft, you will design, build, and maintain AI systems that power some of the largest workloads on the planet. We enable secure, scalable, and high-performance AI experiences across multimodal verticals, including real-time audio interaction, image generation, video generation and safety. Our work spans the entire stack—from API interfaces to inference backends serving AI models—delivering end-to-end solutions that drive innovation at scale.
As part of the Azure OpenAI effort, we collaborate closely with OpenAI and internal partners to advance cutting-edge capabilities. Our team owns critical components such as the safety stack, providing content moderation across different verticals, and real-time APIs, enabling audio-based conversations with large language models through interfaces like SIP, WebSocket, and WebRTC. We benchmark OpenAI and other LLM models for performance on GPUs and Microsoft HW, working together with the inference team to debug and optimize our stacks for reliability and latency.
This is a hands-on technical role requiring deep expertise in distributed systems and AI architectures. We are looking for engineers who thrive on solving complex problems, move fast with autonomy, and embrace collaboration across diverse teams.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day.
Responsibilities
Architect and Implement AI Systems: design and build scalable, reliable architectures that meet high-performance requirements for AI systems including engine, API... Apply deep knowledge of distributed systems principles to create robust solutions.
Debug and Optimize Across the Stack: dive deep into existing components to identify and resolve issues quickly. Ensure fast turnaround for critical fixes while maintaining system integrity.
Collaborate Across Teams and Partners: work closely with multiple internal teams and external partners to solve complex problems, align on technical decisions, and deliver integrated solutions.
Innovate and Drive Technical Excellence: identify non-obvious technical approaches that unlock new possibilities. Architect and execute features with measurable goals, iterating rapidly to achieve outcomes.
Full-Stack Ownership: engage with components across the entire stack—from infrastructure to APIs—to deliver end-to-end solutions.
Leadership and Influence: Articulate complex concepts clearly, build consensus, and create systems that accelerate team velocity. Demonstrate exceptional judgment and autonomy in execution.
Qualifications
Required Qualifications
Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience
5+ years of hands-on experience with designing, building, and maintaining distributed systems with scalable architecture, and API designs.
2+ years of experience AI/ML fundamentals and practical applications, architecting and delivering AI-based solutions.
4+ years of experience with relevant tools and frameworks such as Kubernetes, Azure Devops, or Docker.
Other Requirements
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:
- Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred qualifications
Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
Familiarity with gRPC, FastAPI, Autogen, Semantic Kernel, or LangChain.
Experience with OpenAI API / Azure OpenAI
4+ years of experience in technical leadership, including mentoring and guiding engineers.
Certifications, open-source contributions, published research.
Industry recognition or thought leadership.
#COREAI
#AIPLATFORM
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $139,900 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about **requesting accommodations.**