Name: Introducing Amazon EC2 P6e-GB200 UltraServers: Powering Frontier AI at Scale | Amazon Web Services
Uploaded: 2025-08-30T13:14:33.000Z
Duration: 7 min 59 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

At AWS, it's our mission to offer the best place for AI training and inference.

And doing this means building the best compute infrastructure possible to power the most innovative applications of today and tomorrow.

And that's why I'm super excited to say that NVIDIA Blackwell is now here on AWS.

NVIDIA Blackwell marks a new chapter for accelerated computing.

It delivers significant improvements in performance, cost, and energy efficiency, enabling developers to tackle the world's most challenging problems.

AWS is now bringing the best of NVIDIA Blackwell to the cloud in two system configurations.

Today we're announcing the general availability of both the Amazon EC2 P6 instance, accelerated by NVIDIA's HGX B200, and Amazon's EC2 P6e Ultra Server, accelerated by NVIDIA's GB200

NVL72, our most powerful EC2 instance ever.

Our NVIDIA HGX B200 delivers top performance for x86 scale-up systems.

And our NVIDIA GB200 NVL72 is a rack-scale, liquid-cooled supercomputer that enables 72

NVIDIA Blackwell GPUs to act as a single massive GPU.

These instances will help power training and inference for the latest frontier models, providing up to 30 times greater performance than NVIDIA Hopper.

Blackwell on AWS sets a new bar for performance in EC2, but it's more than meets the eye.

Let's take a closer look at the unique design decisions and innovations we've made to bring

Modern agentic AI workloads are presenting ever-increasing demands for bandwidth in the quest for optimal performance.

This requires solutions for both scaling out and scaling up.

For Blackwell's scaled-out network, we've opted to use our in-house networking stack, the Elastic Fabric Adapter.

Developed by the AWS Nitro system, EFA leverages our latest generation of custom-designed Nitro controllers, allowing us to achieve up to 28.8 terabits per second of networking bandwidth per Ultra server, and scale to EC2 Ultra clusters with up to hundreds of thousands of GPUs.

But Nitro does more than allow us to scale these large clusters.

It allows us to further improve both the reliability and predictability.

And that's because EFA runs over Scalable Reliable Datagram, or SRD, a bespoke, topology-aware networking protocol.

With SRD, Nitro and EFA automatically route traffic around failures on the network, avoiding slowdowns or downtime.

For the scaled-up network, the various GB200 compute trays are connected as a tightly coupled mesh through NVIDIA NVLink, which creates a single 72 Blackwell GPU NVLink domain with 132 terabits per second of total bandwidth, served as a single Ultra server.

To manage these NVLink switches, we leverage our Nitro-powered headnode.

This reduces the number of network devices operating, with the NVLink network managed within the Nitro-trusted domain.

In a fast-moving space, with new platforms going from idea to production faster than ever before, the ability to detect performance issues or bugs is critical for maximizing reliability, security, and performance for customers.

With Nitro, we can gather a greater amount of telemetry, logging everything with respect to function of the hardware, providing us unique insight into detecting and understanding failures.

But the Nitro difference goes beyond detection.

It fundamentally changes the way we apply patches and optimizations.

Typically, infrastructure providers need to choose between keeping their infrastructure online around the clock, forgoing these updates, or taking their capacity offline to update, sometimes resulting in kicking customers off the hardware.

Nitro provides us the capability that no others in the industry have – live updates.

Live updates allow us to update system firmware transparently, while systems remain in operation.

This means that customer workloads continue uninterrupted, with ever-improving reliability, security, and performance, with no downtime required.

Lastly, in order to support the incredible compute density with GB200 NVL72 racks, we've moved towards a new first at AWS – liquid cooling.

Blackwell represents the first liquid cooling hardware platform we've deployed at scale on AWS, and with it comes a unique set of goals and challenges.

This meant meeting the ravenous demand for AI infrastructure, which meant we had to scale fast and scale globally.

When we looked at liquid cooling solutions, based on what's available in the market today, there were a few trade-offs.

First would have been building liquid-cooled data centers from scratch – something that tends to take a while, with long lead times on the order of years.

We knew those timelines wouldn't intersect with the availability of Blackwell.

The second option would have been adopting some off-the-shelf solutions, but these didn't scale.

They would take up too much data center floor space, would still require major modifications to data centers, or increase water usage substantially.

And while some of these solutions could work for lower volumes at other providers, there simply wouldn't be enough liquid cooling capacity to support our scale.

We invented our own solution to have the best of both worlds.

This is the in-row heat exchanger, or IRHX as we call it.

The IRHX is an Amazon-designed liquid cooling system.

It can be installed with no changes to the air-cooled mechanical design and minimal changes to existing infrastructure, meaning it can be deployed in existing data centers as well as new builds.

The IRHX was designed to allow us to scale fast, standardizing our equipment and supply chain, and has been built to spec for our standard rack dimensions to fit within our existing data centers.

There are three components to the IRHX, a water distribution cabinet, a pumping unit, and fan coils.

As the chips in the rack generate heat, cool liquid is pushed from the IRHX pumping unit to the servers.

This liquid is distributed to the chips through a cold plate design co-innovated between AWS and NVIDIA, cooling the chips substantially.

This warm liquid is then returned to the IRHX, being passed through the coils where fans cool the liquid back down while expelling heat out of the back and into an aisle of contained hot air.

And unlike many publicly available options, the IRHX can scale efficiently by adding or subtracting fan coil units depending on the amount of cooling needed by the row.

As we like to say, with the IRHX, we don't need to design the data center around the rack.

With support for over 900 CUDAX libraries and AI models, NVIDIA Blackwell is now available for AWS customers to drive breakthroughs in science, industry, and the next era of AI reasoning.

Blackwell is accessible directly in our P6 instances, accelerated by NVIDIA's HGX B200, and our P6e Ultra servers, accelerated by NVIDIA's GB200 NVL72, or through AWS managed offerings such as Amazon SageMaker HyperPod and NVIDIA DGX Cloud.