Name: How Nvidia GPUs Compare To Google’s And Amazon’s AI Chips
Uploaded: 2025-12-01T23:57:39.000Z
Duration: 15 min 59 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

Nvidia graphics processing units like These latest Blackwell GPUs are inside server racks all over the world.

Participle clauses

Nvidia has catapulted from gaming giant to the very core of generative AI, training the models, running the workloads and sending Nvidia's valuation soaring.

With 6 million Blackwell GPUs shipped over the last year, this connects all 72 GPUs, allowing the ACT as a single GPU to power the most advanced AI workloads.

GPUs are the general purpose workhorse stock of Nvidia and its top competitor amd.

But another big category of AI chip is gaining ground.

The custom ASIC application specific integrated circuit now being designed by all the major hyperscalers.

Google, Amazon, Meta, Microsoft, OpenAI alongside Broadcom.

These AI chips are smaller, cheaper, accessible and as the name suggests, specifically built for one purpose.

But we see that growing even faster than the GPU market over the next few years.

AI chips also include FPGA's Field Programmable Gate arrays and an entire group of chips used to power edge AI on device instead of in the cloud.

In short, there's a lot of AI chips and the space is only getting more crowded.

So let's take a pause to break it all down.

What's different good and bad about them?

Each and a brief look at all the major companies getting in on the AI chip making trend.

For a brief moment in October, Nvidia was the first company ever to reach a $5 trillion valuation thanks to its original gaming engine, the GPU becoming king of AI chips.

GPU in this case was really purpose built to deliver parallel programming because when you think about rendering an image or in a scene, you need to calculate all those pixels at once.

And so AI really lends itself to taking advantage of that capability.

To understand how GPUs became synonymous with AI, let's go back to 2012 to what many consider AI's big bang moment.

Alexnet it was a new, incredibly accurate neural network that obliterated the competition during a prominent image recognition contest.

Turns out, the same parallel processing used by Nvidia GPUs to render lifelike graphics is also great for training neural networks, where a computer learns from data rather than relying on a programmer's code.

The researchers took a GPU and said, I'm going to hack it to get it to expose the parallel computation capabilities to unlock that performance for the deep learning use case.

While AI workloads are usually accelerated on a GPU or custom asic, it still often needs a host central processing unit cpu, like the Grace in Nvidia's Grace Blackwell Server Rack system.

But while a CPU has a small number of powerful cores running sequential general purpose tasks, GPUs have thousands of smaller cores more narrowly focused on parallel math like matrix multiplication, used to process multidimensional data structures known as tensors.

This ability to perform many operations simultaneously makes GPUs ideal for the two main phases of AI training and inference.

Training is teaching the AI model to learn from patterns in large amounts of data, while inference uses the AI to make decisions based on new information.

The way AI shows up in everyday applications, whether it's the way you buy your coffee on your Starbucks app or how you interact with Salesforce for work, or speaking to us through our EarPods, that is being done through inference.

Nvidia sells its GPUs directly to AI companies like in a recent deal to sell at least 4 million to OpenAI and to foreign governments including South Korea, Saudi Arabia and the UK.

It also sells GPUs to cloud providers like Amazon, Microsoft and Google, who go on to rent them out by the hour, minute or second.

Today, systems that can handle AI workloads are in such high demand, One of these 72 Blackwell server racks sells for around $3 million, and

Nvidia told us it's shipping 1,000 each week.

And for Nvidia, they're looking to sell the entire system, not just the chip, because when you think at that system level, you get more efficiencies in terms of speed and power and performance.

Of course, Nvidia isn't the only one making Rackscale GPU systems optimized for AI.

Its top competitor, Advanced Micro Devices AMD has seen major gains with its Instinct GPU line and major commitments from OpenAI and Oracle.

A big difference in AMD GPUs is that they use a largely open source software EcoSystem, while Nvidia GPUs are tightly optimized around CUDA, Nvidia's proprietary software platform.

Nvidia says its next generation Rubin GPU will be in full production next year.

This is just an incredibly beautiful computer.

In the early boom days of large language models, compute intensive training has been key, a perfect job for GPUs, but

as the models mature, more and more inference is needed.

Now post training techniques have been introduced to make these models very capable.

Now to extract the value you need to go to inference and so that's how people actually use AI, is through inference.

Inference can happen on less powerful chips programmed for more specific tasks.

Enter custom asics, application specific integrated circuits and every major cloud provider designing their own.

While a GPU is like a Swiss army knife, able to do many kinds of parallel math for different AI workloads, think of an ASIC like a single purpose tool, very efficient and fast, but

hardwired to do the exact math for one type of job.

They can be much, much more efficient in running those workloads, but

when you specialize them, you can't change them once they're already carved into silicon.

And so there's a trade off in terms of flexibility.

While Nvidia GPUs cost upwards of $40,000 and can be hard to get, startups rely on them because it's even more costly to make your own custom ASIC for AI, it is hugely expensive.

Costs minimum tens and often hundreds of millions of dollars.

But for the biggest cloud providers who can't afford them, custom Asics pay off because they're more power efficient and reduce reliance on Nvidia.

They want to bring cost of AI down, they want to have a little bit more control over the workloads that they build.

But at the same time they're going to continue to work very closely with Nvidia, with amd, because they also need the capacity.

Google was the first to make a custom ASIC for AI acceleration, coining the term Tensor Processing unit when its first ASIC came out in 2015.

The TPU also helped lead to the invention of the transformer at Google in 2017, the architecture powering almost all modern AI.

We got a tour at Google's chip lab in 2024.

There's actually four chips inside there.

Actually two of those are connected to a host and machine that has CPUs in it.

And then all these colorful cables are actually linking together all of the Trillium chips to work as one large supercomputer.

A decade after the first tpu, Google just released Ironwood, its seventh generation and with it a big deal with Anthropic to train its LLM Claude on up to a million TPUs.

Some people even think that they're technically on par or even superior to Nvidia.

But traditionally Google has only used them for in house purposes.

There's a lot of speculation that in the longer run Google might open up access to TPUs more broadly.

Amazon Web Services was the next cloud provider to design its own AI chips after acquiring Israeli chip startup Annapurna Labs in 2015, where Ron Diamant has worked since day one.

AWS announced Inferentia in 2018 and launched Trainium in 2022, now approaching its third generation.

Think of Trainium like a cluster of workshops with many small flexible tensor engines, while Google's TPU is like one big factory conveyor belt with one big rigid grid specialized for matrix math.

The difference between the names is quite telling, but

over time we've seen that Trainium chips can serve both inference and training workloads quite well.

On average we see that trainium provides between 30 and 40% better price performance compared to other hardware vendors in AWS.

In October we went to Northern Indiana for the first on camera tour of Amazon's biggest AI data center, where Anthropic is training its models on half a million Trainium two chips behind me right now they are working on delivering these towers.

Two of them together is going to make up one Ultra server.

And what's especially interesting is there are no Nvidia GPS GPUs in there, although AWS is filling its other data centers with lots of Nvidia GPUs to feed hungry AI customers like OpenAI.

The investment they're making in our platform is still significant and significantly more than you're seeing even in some of the upcoming ASIC deployments.

And we can't talk about custom ASICs without mentioning Broadcom and its top competitor Marvell, which act as backend partners so clients don't have to hire full silicon teams to handle everything in house.

All of the big hyperscalers actually that have ASIC programs partner with at least one chip design company, Broadcom being the most important, which helps provide the IP and the know how and the networking.

And so you've seen Broadcom in particular be one of the biggest beneficiaries of the AI boom.

Broadcom helped build Google's TPUs, Meta's training and inference accelerator launched in 2023 and now a huge new deal to help OpenAI build its own custom ASICs starting in 2026.

We see Broadcom winning 70, maybe even 80% of this market.

We see this market accelerating at a mid double digit CAGR over the next five years.

Microsoft is also getting into the ASIC game, hoping to primarily use its own Maya chips in its Azure data centers, although little has been revealed since plans were announced in 2023 and its next Maya chip is facing delays.

Intel has its own Goudy line of custom Asics for AI.

Tesla has announced an ASIC And Qualcomm is breaking into data center chips with the AI 200.

Of course, there's also a slew of startups going all in on custom AI chips too, like Cerebras making huge full wafer AI chips and Grok with inference focused language processing units.

The final big category of AI chips are those made for running edge AI on device instead of in the cloud.

You might want to be able to run an AI model locally on your personal device, even a smartphone.

And so there may be as part of the processor in that chip a module that has AI processing capability, so you don't have to have communication all the way back to a data center and you can preserve privacy of your data on your phone.

In npu, Neural processing unit is a major edge AI chip, a dedicated AI accelerator that's integrated into a phone or laptop's primary chip, what's called an SoC.

A system on a chip which is kind of a chip that has like many different modules in it that do different aspects of the computation.

Because obviously in a phone you need to have a very compressed system that is able to do lots of things.

And so it's not kind of a separately packaged AI chip.

They take up less silicon and therefore they cost less, often substantially less than a big data center chip would.

NPU's enabling AI and PCs are primarily made by Qualcomm, intel and AMD for MacBooks.

Apple's in house M series chips include a dedicated neural engine too, although Apple doesn't use the term NPU.

Similar neural accelerators, tiny processors dedicated to AI math, are now built into the latest iPhone A series chips too, which we got to see in September.

We know that when we can do things on device, we are able to manage people's privacy in the best way.

The other thing about it is it is efficient for us, it is responsive.

Subtitles ListPlay Video

How Nvidia GPUs Compare To Google’s And Amazon’s AI Chips

specific

process

dedicated

acquire