- AWS built custom Nvidia cooling after rejecting existing liquid solutions for scale
- IRHX fits into AWS racks without changes to existing infrastructure
- Amazon could extend this cooling approach to Graviton chips in the future
Amazon Web Services (AWS) has introduced a proprietary cooling system built to handle the demands of Nvidia’s newest GPUs.
The In-Row Heat Exchanger, or IRHX, was developed in response to the growing power and heat requirements of hardware like the Nvidia GB200 NVL72.
AWS evaluated existing liquid cooling solutions but found they didn’t fit the company’s needs.
You may like
AWS Graviton next?
“They would take up too much data center floor space, would still require major modifications to data centers, or increase water usage substantially,” Dave Brown, VP Compute and ML Services at AWS, said in a presentation posted on YouTube, which you can see below.
“And while some of these solutions could work for lower volumes at other providers, they simply wouldn’t be enough liquid cooling capacity to support our scale.”
The IRHX system consists of a pumping unit, a water distribution cabinet, and fan coils.
Liquid cools the chips through a cold plate co-designed by AWS and Nvidia, then cycles back through the IRHX, where it is cooled and released.
“With the IRHX we don’t need to design the data center around the rack,” Brown said.
The system supports AWS’s most powerful EC2 instance, the P6e UltraServer, which includes the GB200 NVL72. This rack-scale setup allows 72 Blackwell GPUs to work together as one unit.
Brown said the GB200 NVL72 “enables 72 Nvidia Blackwell GPUs to act as a single massive GPU.”
Amazon has previously built custom hardware, including chips and networking systems. The IRHX extends that strategy into cooling, allowing AWS to deploy new GPU racks without redesigning its facilities.
The company said the system fits existing rack dimensions and infrastructure, making it scalable across global data centers.
While IRHX is currently paired with Nvidia’s Blackwell-based systems, it is likely to be used with Amazon’s own Graviton chips if their cooling needs rise.
For now, the system is powering AI workloads that demand both scale and speed.
Introducing Amazon EC2 P6e-GB200 UltraServers: Powering Frontier AI at Scale | Amazon Web Services – YouTube
Watch On
