Senior System Software Engineer - AI Data Platform - Inference Factory Optimization

NVIDIA NVIDIA · Semiconductors · Hanoi, Vietnam +1 · Remote

This role focuses on building and optimizing the infrastructure for deploying and optimizing AI inference workloads. The engineer will develop automation systems, performance optimization tools, and work on system-level programming to ensure efficient deployment of AI models and applications.

What you'd actually do

  1. Develop efficient infrastructure and tools for automating complex software processes.
  2. Implement advanced test harnesses, benchmarking frameworks, and analytical tools to rigorously characterize and optimize the performance and efficiency of our software and hardware platforms.
  3. Apply deep knowledge of operating systems, kernel internals, device drivers, memory management, storage, networking, and high-speed interconnects to build and troubleshoot highly performant systems.
  4. Work with engineering teams to understand needs, define requirements, and deliver efficient solutions.
  5. Set performance goals, monitor feedback, analyze data, and make continuous improvements for system reliability.

Skills

Required

  • 5+ years of industry experience in software development, focusing on infrastructure, distributed systems, automation, and/or performance engineering.
  • Expertise in System-Level Programming: Proven ability to develop robust tools and automation using programming languages such as C++, Python, or Go.
  • Deep Understanding of System Software: Experience with operating system internals, device drivers, memory management, and debugging performance issues in complex compute applications.
  • Distributed Systems: Experience in designing, building, and operating large-scale distributed systems, with knowledge of networking protocols, cluster management, and high-performance interconnects.
  • Automation and CI/CD Proficiency: Experience building and maintaining automated testing, benchmarking, and continuous integration/continuous deployment pipelines.
  • Problem-Solving and Analytical Skills: Outstanding analytical, problem-solving, and debugging skills, with a track record of resolving complex technical challenges.
  • Collaboration and Communication: Excellent interpersonal and communication skills, with the ability to articulate complex technical concepts to diverse audiences and collaborate effectively across teams.

Nice to have

  • Experience optimizing performance for AI/Machine Learning workloads, especially inference applications, on diverse hardware platforms.
  • Prior experience building or contributing to large-scale compute infrastructure solutions in cloud environments or on-premises data centers.
  • Experience with containerization and orchestration technologies, such as Docker and Kubernetes.
  • Familiarity with performance profiling tools and methodologies for hardware and software systems.
  • Track record of driving significant efficiency gains or architectural improvements in large-scale systems.

What the JD emphasized

  • performance optimization
  • AI/Machine Learning workloads
  • inference applications

Other signals

  • optimizing inference performance
  • building foundational infrastructure for AI
  • automating deployment of AI models