Member of Technical Staff - Pre-training

xAI xAI · AI Frontier · Palo Alto, CA · Model

xAI is seeking a Member of Technical Staff focused on Pre-Training to train trillion-parameter neural networks at scale. The role involves implementing state-of-the-art methods, innovating in pretraining and scaling paradigms, and requires strong engineering skills in model-hardware co-design, ML scaling laws, and distributed training for efficiency.

What you'd actually do

  1. Training trillion parameter neural networks at scale, as well as a variety of smaller specialized models.
  2. Rapidly implementing the latest state-of-the-art methods from the deep learning literature.
  3. Innovating new ideas for pretraining and new scaling paradigm.

Skills

Required

  • Strong engineering skills
  • model-hardware co-design
  • Expert in ML and large model scaling
  • familiar with all kinds of scaling laws
  • Familiar with distributed training
  • multi-GPU neural network training
  • experience on optimizing ML training efficiency

What the JD emphasized

  • trillion parameter neural networks
  • latest state-of-the-art methods
  • pretraining
  • scaling laws
  • distributed training
  • ML training efficiency

Other signals

  • trillion parameter neural networks
  • latest state-of-the-art methods
  • innovating new ideas for pretraining
  • scaling laws
  • distributed training
  • ML training efficiency