Name: Interconnect Options for Ethernet Scale Up Networks
Uploaded: 2026-03-28T10:49:42.000Z
Duration: 23 min 21 s
Description: Thousands of YouTube videos with English-Chinese subtitles! Now you can learn to understand native speakers, expand your vocabulary, and improve your pronunciation...

So, um, welcome to the second half of the Shorelish Optics Interconnects workshop, and, uh, we s we have five more speakers, uh, in the strong lineup, uh, to continuously discuss about the Scale Up Network, how optics and Scale Up Network can interact together.

Um, so, you know, without s-speaking more, welcome, Peter.

Can we start it now or wait a couple minutes?

Um, I think we can s-start it a little bit earlier if, uh that, that would that would give, uh, more, more time for, for questions.

I manage the Tomahawk product line at Broadcom.

So the impetus for this talk was actually a suggestion by Cliff Grossner a few months back that I provide an overview about SUE, or Scale Up Inter-Interconnect, or Scale Up Ethernet, to the Short-Reach Optical Interconnect group.

So over the last few months, a lot has changed, so actually I've reflected that in the presentation.

Um, over the last few months, we've announced Tomahawk Ultra, which is a chip that's focused on HPC and scale up.

It's actually a proof point that we can actually achieve very high performance.

It's actually sub-250 nanoseconds latency for scale up based on Ethernet.

One other thing that was announced just earlier this week is the ESUN, or Ethernet for Scale Up Networking, initiative.

I've got some slides on what is SUE, or what's evolved now into SUE Transport, and

Okay, so first I'll provide some context.

You know, one of my motivations for this is that managing the Tomahawk product line, I wanna make sure there's an e is a full ecosystem of i-interconnect for these switches.

So I wanna make sure there's electrical interconnect.

I wanna make sure there's optical interconnect.

So as mentioned before, we already have Tomahawk Ultra.

We have customers that already demonstrated systems based on this.

So there's a proof point that you can do scale up with Ethernet, with sub-250 nanoseconds latency.

So we actually have two versions of Tomahawk 6.

We have one that's based on 200 gig SERDES.

I brought the, uh, cargo pants here 'cause, uh, it's a rather large package.

So this is, I think, the largest package on the market right now for a networking chip.

So we do have customers that are gonna be deploying Tomahawk 6 and Tomahawk Ultra next year, both for scale up and for scale out.

So one of the key things here is that, you know, with these chips, we can already achieve a scale out d a scale up domain of two of 512 XPUs.

And one of the things I think we need to look at going forward is what are the optics that will be needed for these switches, the switches we'll have in the future, especially considering the variety of packaging options that will be available.

And I'll get into the different packaging options we're considering on future slides.

And also to provide some more context, at Broadcom, we do have a full end-to-end suite of products for scale up networking.

So on the switches, we have Tomahawk Ultra, Tomahawk 6.

So we have discrete NIC, you know, products.

We have NIC chiplets that we provide to our customers that use our custom ASIC services.

And if customers want to roll their own IP, we provide a set of specifications about how they can do scale up efficiently using Ethernet.

And of course, we have ASICs, so a large number of custom ASICs.

So, you know, important point here is that when we're looking at the optics, we have to make sure they're gonna work for all these endpoints.

it's gotta be all these components that it works end to end.

So if we look at the trends for the scale up cluster sizes, what we're seeing today is 64, 72 GPUs.

Very soon, we're gonna see deployments of 256 to 512 GPUs, or XPUs.

And we already have a lot of requests for much larger scale up domains, you know, 1,000 to 2,000 XPUs.

So an important point here is that this is not just it's no longer a box level.

This is something that it needs to be integrated with the rest of the DC infrastructure.

So if you use some bespoke technologies, proprietary technologies, it's not gonna scale.

You're gonna wind up also having different operational tools throughout the data center.

So we do have to consider that, you know, in the past, you ha you the, the narrative was that you had Ethernet for your wide area data center network, maybe used in FinnaBand for the scale out, and

then some proprietary technology for the scale up.

You know, right now, I think people have concluded that, yes, we wanna use Ethernet for the wide area, for scale out.

And I think the next thing, especially since these scale up domains are increasing so much in size, is to put Ethernet into the scale up domains.

So some of the benefits you get from that is you have a common technology stack, common operational tools.

So you don't have to have separate network engineers that understand different technologies.

You can use the same switches, the same cables, the same optics.

One other thing that we hear is that the hyperscalers, if they're deploying a very large data center, what they would like to do is have fungible interfaces.

So if they have a data center that's got, say, 64K or 128K XPUs, a customer renting some XPU time from them might not rent out that whole data center.

So they might wanna be able to partition that where they wind up having, you know, a smaller cluster for a certain user.

And they might also wanna set it up so that way they have different interfaces for scale up versus scale out in different parts of that DC.

And the only technology that can do that is Ethernet.

So as far as the requirements, um, I think folks are most are aware that you need to have high bandwidth, very efficient data transfer.

You also wanna have very reliable connectivity.

I mean, in general, over the scale up interfaces, this is gonna be a memory transaction.

So you want that to behave like a memory transaction, to be lossless.

Um, going along with that, you also wanna make sure you have very efficient on-chip IP.

So if you have, say, 10 terabits per second or more of scale up bandwidth, you wanna make sure that's not taking up a very large part of your XPU real estate and lo large part of the power.

So in order to do this, you wanna make sure that you have an Ethernet implementation, you know, an IP that's really optimized for scale up.

So as far as bandwidth requirements, what we're seeing today is that the XPUs are evolving from, say, four HBM stacks to H-HBM stacks with 12.8 terabits per second.

So soon you'll see that the HBM bandwidth will be over 100 terabits per second.

And what we're seeing in general is that the scale up bandwidth tends to be 10 to 20 percent of that.

I mean, effectively, you can see that scale up interface as being memory sharing or, you know, memory transactions between the different XPUs.

So that's roughly what we're seeing as far as the ratio, maybe like a 10 to 1, you know, ratio of HBM bandwidth to scale up bandwidth.

So overall, as far as the system bandwidth requirements, what we're seeing today is XPU clusters on the order of 10, you know, tens of thousands of XPUs with 10 terabits per second of scale up bandwidth, 800 gig of scale out bandwidth.

Uh, we see those clusters going to a million plus XPUs.

As far as the scale up bandwidth, we already have requests from hyperscalers that are saying literally that in, like, the next three to five years, they wanna see 50 to 100 terabits per second of just scale up bandwidth from an XPU.

So net result is, you know, 100 million terabits per second of total bandwidth.

So, you know, huge, huge amount of bandwidth.

We need to be ex as efficient as possible on the optics.

So as far as the interconnect options, um, you know, copper is king.

You know, wherever you can use copper, you will.

You know, we still see this being viable at 400 gig per lane.

Um, you know, you may need to use some retimers or redrivers in some cases, but

definitely we see that copper is still gonna be very viable at 400 gig.

Um, so this will always be the preferred option.

Subtitles ListPlay Video

Interconnect Options for Ethernet Scale Up Networks

process

scale

common

general