Anyscale

Anyscale is seeking a Distributed LLM Inference Engineer to build and optimize systems for large-scale LLM inference, integrating with Ray and open-source projects like vLLM. The role focuses on achieving high throughput and low latency for batch and online inference, contributing to Anyscale's market-leading AI infrastructure.

Software Engineer role focused on building and scaling the platform infrastructure for distributed AI applications using Ray. Responsibilities include control plane and data plane development, Kubernetes, container orchestration, and cloud-native infrastructure. The role involves optimizing performance, reliability, and observability of the platform.