Getting Started with SR-IOV for High-Speed Networking on Hyperstack

We’re excited to announce the release of on-demand high-speed networking with SR-IOV. This is a significant upgrade for our users as we provide high-performance networking for workloads that demand increased bandwidth between virtual machines, often referred to as inter-VM bandwidth. Hyperstack provides GPU VMs with high-speed networking capabilities powered by SR-IOV (Single Root I/O Virtualisation) technology. SR-IOV allows a single physical network interface to be shared across multiple VMs, supporting inter-VM bandwidth of up to 350 Gbps, depending on the thread count and MTU size.

With the release of this feature, we ensure that your AI workloads will perform at maximum efficiency by optimising data transfers and reducing latency. To get started with our latest SR-IOV feature, continue reading.

Understanding GPU Support for High-Speed Networking

Before getting started with SR-IOV, it’s important to know which GPUs support high-speed networking. Hyperstack offers the following NVIDIA GPUs that support high-speed networking:

• A100 PCIe with NVLink
• H100 PCIe
• H100 PCIe with NVLink

These GPUs leverage SR-IOV technology through the PCIe interface to provide direct, high-speed access to GPU resources and reduce overhead and latency. The NVLink feature further improves communication between GPUs which makes them ideal for tasks requiring high throughput and low latency.

How to Use SR-IOV for High-Speed Networking?

Virtual Machines with network-optimised configurations in the legacy CANADA-1 environments offer inter-VM bandwidth of up to 125 Gbps. To enhance networking performance and achieve up to 350 Gbps bandwidth, follow the steps below to set up Hyperstack's high-speed networking with SR-IOV technology:

Step 1: Create a network-optimised environment

1. In Hyperstack, go to the Environments page.
2. Click Create New Environment.

• Enter a name for the environment.
• Select the CANADA-1 region, where high-speed networking flavours are available.

• Click Create Environment:

Step 2: Deploy a network-optimised VM within a network-optimised environment

1. In Hyperstack, go to the VM deployment page by clicking Virtual Machines on the sidebar, then click the Deploy New Virtual Machine button.

• Choose GPU Flavour: Choose a network-optimised Hyperstack flavour, such as NVIDIA A100 with NVLink, NVIDIA H100 or NVIDIA H100 with NVLink.

• Select Environment: Select a network-optimised environment in the CANADA-1 region, indicated by the⚡ icon.

• Select OS Image: Choose Ubuntu Server 22.04 LTS R535 CUDA 12.2 image for your VM's operating system, as high-speed networking is exclusively available with this Ubuntu image.

• Configure any other VM features as needed.

Step 3: Deploy Your Configured VM

After configuring your VM as desired, click Deploy. Your network-optimised VM with inter-VM bandwidth of up to 350 Gbps has been successfully deployed when it is marked with the ⚡ icon.

Who Can Use SR-IOV?

The SR-IOV feature is released to address users who rely on high-performance computing and need to maximise the efficiency of their resources. The following groups will benefit most from this release:

• Data Scientists and AI Engineers: For those working with large-scale AI model training, our SR-IOV feature accelerates data transfer between GPU VMs, optimising model training and improving GPU utilisation.
• Real-Time Analytics and Financial Services: Companies requiring real-time data analytics with minimal latency can use our SR-IOV feature to streamline data transfers and improve responsiveness.
• High-Performance Computing (HPC) Users: Researchers and institutions using HPC workloads will highly benefit from our SR-IOV’s ability to reduce latency, increase throughput and fully utilise computing resources.

What are the Benefits of SR-IOV?

The SR-IOV feature has multiple benefits, including:

1. Enhanced Network Throughput: SR-IOV delivers network throughput that approaches line rate for high-speed data transfer between VMs. This significantly boosts performance, particularly for workloads that rely heavily on large data sets and real-time processing.
2. Accelerated AI Training: With SR-IOV's performance-speed networking, data transfer between GPU VMs becomes significantly faster. This minimises the delays often faced during large-scale AI model training and allows models to train quicker and reduce total time to completion.
3. Optimised GPU Utilisation: SR-IOV’s ability to share PCIe devices among VMs efficiently ensures that your GPUs are used to their full potential. GPU utilisation is significantly improved by minimising idle time and maximising data flow, resulting in faster task completion.
4. Reduced Data Transfer Latency: SR-IOV minimises the latency typically experienced in data transfers between VMs, ensuring faster response times. This is crucial for tasks like real-time analytics or complex AI model training, where rapid data availability is key to quick convergence and accurate results.

Conclusion

The release of SR-IOV on Hyperstack ensures that users with latency-sensitive workloads now have access to the highest level of network performance. It does not matter if you’re training AI models, conducting real-time analytics or working on other demanding computing tasks, the SR-IOV provides the networking power to help you achieve your goals faster and more efficiently. Stay tuned for the next two parts of our SR-IOV series, where we’ll continue exploring how to optimise SR-IOV for AI and high-performance cloud applications- coming next week!

Subscribe to our newsletter below to receive product updates directly in your inbox as we aim to deliver the best solutions for you!

FAQs

What is SR-IOV?

SR-IOV (Single Root I/O Virtualisation) allows a single physical network interface to be shared across multiple virtual machines, providing high-speed, low-latency networking.

Which GPUs support SR-IOV on Hyperstack?

Hyperstack supports SR-IOV with NVIDIA GPUs such as NVIDIA A100 PCIe with NVLink, NVIDIA H100 PCIe and NVIDIA H100 PCIe with NVLink.

How can I create a high-speed networking environment?

To create a high-speed networking environment, go to the Environments page in Hyperstack, click "Create New Environment," and select the CANADA-1 region.

What OS image is required for high-speed networking?

For high-speed networking, you should use the Ubuntu Server 22.04 LTS R535 CUDA 12.2 image, as it's the only supported image for this feature.

How do I know if my VM deployment is successful?

Your network-optimised VM is successfully deployed when it is marked with the ⚡ icon, indicating that it has achieved inter-VM bandwidth of up to 350 Gbps.