About the role
As a Forward Deployed Engineer (FDE) focused on large scale GPU clusters, you will be a hands-on technical partner to our strategic customers – the world’s leading AI model builders. You will partner with our SAs as a deep-domain specialist in large-scale infrastructure, storage, high-performance networking, and cluster orchestration. As key contributors to the CX, Engineering, and Sales organizations, FDEs add tremendous value by ensuring we can meet the requirements of our most complex POCs, facilitate successful platform adoption for our strategic customers, and guide tailored optimization efforts - directly impacting company growth and the hardening of our core platform.
Responsibilities
- Cluster Hardening & Validation: Design and execute rigorous pre-handover test suites (NCCL, DCGM, GPU Burn) to ensure clusters are stable under the extreme stress of multi-node training.
- Technical Partnership: Act as the primary technical point of contact for model labs, helping them tune their orchestration layer (Kubernetes or SLURM) for maximum throughput.
- Infrastructure Optimization: Profile and debug low-level bottlenecks in InfiniBand (IB) fabrics, NVLink topologies, and high-performance storage systems.
- Opinionated Onboarding: Build reference designs and "out-of-the-box" configurations for training frameworks to reduce customer time-to-train.
- Benchmarking & Migration: Lead complex benchmarking exercises to demonstrate the performance impact of migrating to new hardware families or Together AI’s optimized infrastructure.
- Product Feedback Loop: Directly influence our hardware and software roadmap by surfacing edge cases and performance gaps found during customer deployments.
Requirements
- Experience: 5+ years in a technical role, with a strong focus on Large-Scale GPU Infrastructure.
- Orchestration Mastery: Deep, hands-on experience with Kubernetes (specifically GPU-operator and device plugins) and/or SLURM for workload scheduling.
- Networking & Interconnects: Expert knowledge of InfiniBand, RoCE, and NVLink; ability to diagnose network failures that degrade collective communication (NCCL).
- Storage Knowledge: Familiarity with parallel file systems (VAST or Weka preferred) and object storage, specifically in the context of large-scale checkpointing.
- Benchmarking Skills: Ability to run and interpret training benchmarks and communication tests to validate cluster health and performance.
- Coding & Automation: Proficiency in Python and shell scripting; experience with Ansible or similar tools for automated cluster configuration.
- Willingness to dive into the customer's stack to solve hard problems and comfortable with the high-stakes, fast-paced environment of frontier model labs.
About Together AI
Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society, and together we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI, and our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers on our journey in building the next generation of AI infrastructure.
Compensation
We offer competitive compensation, startup equity, health insurance, and other benefits, as well as flexibility in terms of remote work. The US base salary range for this full-time position is: $270,000 - $300,000 OTE + equity + benefits. Our salary ranges are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
Please see our Privacy Policy at https://www.together.ai/privacy