5 Interesting Facts About the NVIDIA H100 GPUs

When NVIDIA CEO Jensen Huang called the H100 the” engine of the world's AI infrastructure", he wasn’t just making a statement, it was him showing us the future of AI at scale. Released as part of the groundbreaking Hopper architecture, the NVIDIA H100 has become the go-to solution for enterprises looking to accelerate their AI-driven businesses. Leading companies like Meta and Tesla are using NVIDIA H100 GPUs and we can already witness how they are leading the future of AI technology. 

5 Facts About NVIDIA H100 GPUs

Continue reading as we explore 5 fascinating facts about H100 GPUs below!

1. Meta and Tesla Leading the Race

Did you know Meta and Tesla are among the leading consumers of NVIDIA H100 GPUs? Earlier this year, Meta announced its plan to expand its Generative AI infrastructure by the end of 2024 with 350,000 H100 GPUs and additional systems that will collectively deliver compute power comparable to nearly 600,000 H100s. This makes Meta the largest consumer of NVIDIA’s H100 GPUs. 

Meanwhile, Tesla’s xAI Colossus supercomputer based in Memphis operates with 100,000 NVIDIA H100 GPUs using the NVIDIA Spectrum-X Ethernet networking for high-performance AI workflows. Colossus supports training Tesla’s Grok LLMs including chatbots for X Premium subscribers. It's exciting to hear that Tesla is planning to boost Colossus' capacity to an impressive 200,000 H100 GPUs, aiming to create the world's largest AI supercomputer!

2. Build Secure AI Models

NVIDIA is not just leading the AI innovation, it is also building secure AI systems. The NVIDIA H100 is the first GPU to feature confidential computing capabilities. This is nothing short of a revolutionary technology because it ensures your sensitive data remains secure even during processing. During AI training or inference, it is important to protect both the data and the code as input data frequently contains personally identifiable information (PII) or critical enterprise secrets. And the trained models are even more valuable intellectual property (IP). 

But how does it work? NVIDIA H100's Confidential Computing secures data and code during processing using hardware-based isolation and trusted execution environments (TEEs). This ensures sensitive data and AI models are protected from unauthorised access or modification while being used.

It's no surprise that H100 GPUs are the go-to choice for secure AI operations in important fields like finance, healthcare and defence!

3. The Performance is at Scale

You know it's NVIDIA when it comes to scaling AI. The NVIDIA H100 is built for exceptional individual performance and unmatched scalability. Enterprises can deploy up to 256 GPUs in a single cluster, achieving extraordinary compute power for workloads like weather simulations or training multi-trillion parameter AI models.

The NVIDIA H100 GPUs are built with 18 fourth-generation NVLink connections, facilitating high-bandwidth communication between GPUs. This design ensures that even large clusters can operate with minimal latency.

No matter how big or small your organization is, you can always count on the H100s. Whether you're training a single model or scaling up to global applications, the H100 GPUs are setting a new benchmark for AI innovation.

4. Future of AI Infrastructure

Did you know the NVIDIA DGX System is the first-ever AI platform designed with NVIDIA H100 Tensor Core GPUs? Each system includes eight H100 GPUs connected as one using NVIDIA NVLink, delivering an impressive 32 petaflops of AI performance at FP8 precision- six times the power of its predecessor.

“AI has fundamentally changed what software can do and how it is produced. Companies revolutionizing their industries with AI realize the importance of their AI infrastructure. Our new DGX H100 systems will power enterprise AI factories to refine data into our most valuable resource — intelligence.”

- Jensen Huang, founder and CEO of NVIDIA.

This system is designed for the most demanding tasks like training large language models, accelerating healthcare research and tackling climate science challenges. This system has led the foundation for next-gen AI infrastructure.

And that’s not all, with 576 DGX H100 systems and a total of 4,608 H100 GPUs, NVIDIA will launch the NVIDIA Eos that will become the world’s fastest AI system when it goes live later this year. The NVIDIA Eos is said to be four times faster in AI processing than Japan’s Fugaku supercomputer, the current speed leader.

5. World's Most Flexible Sound System

NVIDIA has rolled out Fugatto, a foundational generative transformer model that builds on the NVIDIA team's earlier breakthroughs in areas like speech modelling, audio vocoding and audio comprehension. This impressive model uses 2.5 billion parameters and was trained on a cluster of NVIDIA DGX systems, each packed with 32 NVIDIA H100 Tensor Core GPUs. It is interesting to see how the NVIDIA H100 GPUs power the world's most flexible sound machine system. 

6. Bonus Fact: Beating the NVIDIA A100

The NVIDIA A100 is indeed revolutionary, but the H100 is built for the future. The NVIDIA A100 introduced us to tensor float-32 (TF32) precision and scalable multi-GPU setups, the H100 improves performance by up to 3x in AI training and 6x in inference tasks. Thanks to the Hopper architecture, the H100 supports Transformer Engine optimisations, making it ideal for large language models like Llama 3. With higher memory bandwidth and 18 fourth-generation NVLink connections, the H100 GPUs offer better scalability for large AI clusters.

Had fun discovering the NVIDIA H100 GPUs? Stick around for our next blog where we'll dive into 5 surprising use cases of NVIDIA A100 GPUs you might not have heard about!

Try NVIDIA H100 GPUs on Hyperstack today, starting at $1.33/hour!

FAQs

What is the cost of NVIDIA H100 GPUs at Hyperstack?

The cost of NVIDIA H100 GPUs at Hyperstack starts at $1.33/hour, offering both PCIe and SXM options. Check out our cloud GPU pricing here!

Can I get NVIDIA H100 GPUs on Hyperstack?

Yes, you can access NVIDIA H100 PCIe and NVIDIA H100 SXM GPUs on Hyperstack for high-performance AI workloads. Sign up here to access NVIDIA H100 GPUs now: 

How do I deploy NVIDIA H100 GPUs on Hyperstack?

Hyperstack offers simple 1-click deployment for NVIDIA H100 GPUs, making it easy to get started with AI models, training and inference. Watch Hyperstack GPU Cloud Platform Quick Tour to get started.