RayTurbo

Product · API / developer platform by Anyscale

The supercharged, GPU-native Ray engine for production AI.

RayTurbo is an optimized Ray runtime for AI teams, offering up to 4.5X faster data processing and 90% lower costs through GPU-native architecture and spot instance support. It is available exclusively on Anyscale and integrates natively with major Kubernetes services.

Launched
2024-10-01
Languages
Python

Category: AI Infrastructure

Data processing speedup
Up to 4.5X
Serving QPS improvement
Up to 60%
Node launch speedup
Up to 5X
Cost reduction (spot instances)
Up to 90%
Node reduction for serving
Up to 50%
Architecture type
GPU-native with static task graphs
  • Up to 4.5X faster data processing than open-source Ray
  • 60% higher QPS for model serving workloads
  • Reduces costs by up to 90% with reliable spot instance support
  • 5X faster node launching and scaling for AI clusters

Overview

RayTurbo is an optimized, high-performance version of the Ray runtime available exclusively on the Anyscale platform. It is designed for businesses and AI teams that need to scale distributed training, multimodal data processing, and model serving with greater efficiency than open-source alternatives. The engine features a GPU-native architecture that utilizes static task graphs to reduce overhead and improve inter-GPU communication speed. RayTurbo delivers significant performance gains, including up to 4.5X faster data processing and 60% higher queries per second for model serving. It is natively supported on Google Kubernetes Engine (GKE) and can be deployed across major cloud providers via the Anyscale Operator for Kubernetes. By maximizing resource utilization and providing reliable spot instance support, it can reduce AI infrastructure costs by as much as 90%.

Performance and Speed

  • 4.5X Faster Data Processing: RayTurbo delivers up to 4.5X faster data processing compared to open-source Ray.
  • 60% Higher Serving QPS: The engine achieves up to 60% higher queries per second (QPS) for model serving workloads.
  • 5X Faster Scaling: RayTurbo enables up to 5X faster node launching and scaling for AI clusters.
  • GPU-Native Architecture: The architecture uses static task graphs to set a new standard in speed and low latency for inter-GPU communication.

Cost Efficiency

  • 60% Cost Reduction: RayTurbo reduces infrastructure costs by as much as 60% through optimized resource utilization.
  • 90% Lower Spot Costs: The platform provides reliable support for spot instances, potentially lowering costs by up to 90%.
  • Resource Maximization: The engine maximizes resource utilization to minimize additional costs for large-scale AI development.
  • 50% Fewer Nodes: RayTurbo can power online model serving with up to 50% fewer nodes.

Deployment and Integration

  • Google Cloud Integration: RayTurbo is supported natively on Google Kubernetes Engine (GKE) for integrated cluster management.
  • Multi-Cloud Kubernetes Support: The Anyscale Operator for Kubernetes allows users to deploy RayTurbo within existing Amazon EKS, Azure AKS, and OCI clusters.
  • Unified Workload Support: The engine is designed to handle multimodal data processing, distributed training, and model serving.

Use cases

  • AI teams use RayTurbo to process multimodal datasets up to 4.5X faster for foundation model training.
  • Machine learning engineers deploy RayTurbo on GKE to achieve 60% higher throughput for real-time model serving.
  • Infrastructure leaders use RayTurbo's spot instance support to reduce the cost of large-scale training jobs by up to 90%.
  • Developers scale AI workloads from local environments to thousands of nodes with 5X faster cluster launching.

Specifications

Deployment
Available exclusively on the Anyscale platform; supported on GKE, EKS, AKS, and OKE via Anyscale Operator
Integrations
Google Kubernetes Engine (GKE), Amazon EKS, Azure AKS, OCI Kubernetes Engine (OKE), PyTorch, vLLM
Data processing speedup
Up to 4.5X
Serving QPS improvement
Up to 60%
Node launch speedup
Up to 5X
Cost reduction (spot instances)
Up to 90%
Node reduction for serving
Up to 50%
Architecture type
GPU-native with static task graphs

How it works

  1. Access the Platform: Users sign up for the Anyscale platform to access the exclusive RayTurbo engine.
  2. Deploy the Operator: Deploy the Anyscale Operator into existing Kubernetes clusters on AWS, Azure, Google Cloud, or OCI.
  3. Execute Workloads: Run distributed training, data processing, or serving jobs using Ray's Pythonic APIs with the optimized RayTurbo runtime.

FAQ

What is RayTurbo?

RayTurbo is an optimized version of the Ray runtime available exclusively on the Anyscale platform, designed for peak performance and GPU-native execution.

How much faster is RayTurbo than open-source Ray?

It delivers up to 4.5X faster data processing, 60% higher QPS for serving, and 5X faster node scaling compared to the open-source version of Ray.

Can I use RayTurbo on my existing Kubernetes cluster?

Yes, through the Anyscale Operator for Kubernetes, RayTurbo can be deployed on Google GKE, Amazon EKS, Azure AKS, and OCI OKE.

What are the cost benefits of using RayTurbo?

RayTurbo reduces costs by up to 60% through higher efficiency and up to 90% through reliable support for spot instances.

Start for Free

Added by AntSpace from public sources (anyscale.com).

Related

More products by Anyscale