Anyscale

Anyscale is seeking a Distributed LLM Inference Engineer to build and optimize systems for large-scale LLM inference, integrating with Ray and open-source projects like vLLM. The role focuses on achieving high throughput and low latency for batch and online inference, contributing to Anyscale's market-leading AI infrastructure.

Software Engineer role focused on building and scaling Ray Serve, a production-grade serving framework for machine learning applications. The role involves developing high-performance, distributed systems for ML model deployment, including asynchronous inference, sub-millisecond routing, zero-downtime updates, state management, multi-model orchestration, and observability tools. It requires strong systems programming, distributed systems, and cloud-native infrastructure knowledge, with a focus on production reliability and performance.

Software Engineer role focused on building and scaling the platform infrastructure for distributed AI applications using Ray. Responsibilities include control plane and data plane development, Kubernetes, container orchestration, and cloud-native infrastructure. The role involves optimizing performance, reliability, and observability of the platform.

Software Engineer role focused on the core backend of Ray, an open-source project for distributed computing and scalable machine learning. Responsibilities include developing high-quality open-source software, implementing architectural improvements, enhancing testing infrastructure, and communicating work. Requires experience in distributed systems, algorithms, and data structures. Knowledge of distributed model training/inference and GPU programming is preferred.

Software Engineer to build, optimize, and scale Ray's Datasets library and data processing capabilities, focusing on performance, integration with ML libraries, and stability of distributed systems for machine learning.

Security Engineer responsible for architecting and building secure products, establishing and implementing security best practices and policies, and supporting compliance certification efforts.