Would you like to inspect the original subtitles? These are the user uploaded subtitles that are being translated:
1
00:00:00,000 --> 00:00:02,100
it's pretty clear where nvidia's
2
00:00:02,101 --> 00:00:04,250
priorities lie these days we're here at
3
00:00:04,251 --> 00:00:05,700
the computex booth of one of their
4
00:00:05,701 --> 00:00:09,120
Partners gigabyte and this is the entire
5
00:00:09,121 --> 00:00:11,460
gaming showcase
6
00:00:11,461 --> 00:00:13,740
that's because they like the rest of the
7
00:00:13,741 --> 00:00:15,830
industry understand that the future of
8
00:00:15,831 --> 00:00:19,440
computing lies in the data center
9
00:00:19,441 --> 00:00:23,100
that is where the grace Superchip comes
10
00:00:23,101 --> 00:00:26,160
in under each of these gigantic heat
11
00:00:26,161 --> 00:00:30,110
spreaders are 72 of nvidia's Grace CPU
12
00:00:30,111 --> 00:00:32,340
course connected together using what
13
00:00:32,341 --> 00:00:35,100
Nvidia calls the Envy link chip to chip
14
00:00:35,101 --> 00:00:37,520
interconnect for a total of
15
00:00:37,521 --> 00:00:41,820
144 cores except that's just one of the
16
00:00:41,821 --> 00:00:44,820
nodes This Server from gigabyte accepts
17
00:00:44,821 --> 00:00:48,600
not one not two but four of these
18
00:00:48,601 --> 00:00:51,710
modules in its four separate nodes that
19
00:00:51,711 --> 00:00:54,680
is an absolutely mind-bending
20
00:00:54,681 --> 00:01:00,000
576 cores in a 2u server rack but these
21
00:01:00,001 --> 00:01:02,520
are not the types of CPUs that you have
22
00:01:02,521 --> 00:01:04,440
in your gaming PC at home those
23
00:01:04,441 --> 00:01:07,200
processors from the likes of AMD and
24
00:01:07,201 --> 00:01:09,970
Intel are based on the x86 architecture
25
00:01:09,971 --> 00:01:13,320
so similar to what Apple did with their
26
00:01:13,321 --> 00:01:17,039
M series M1 and M2 processors Nvidia is
27
00:01:17,040 --> 00:01:19,080
making use of a different processor
28
00:01:19,081 --> 00:01:21,470
architect picture called arm
29
00:01:21,471 --> 00:01:24,420
and uh we actually did get permission to
30
00:01:24,421 --> 00:01:26,280
do this we're going to be taking a
31
00:01:26,281 --> 00:01:28,930
closer look here
32
00:01:31,560 --> 00:01:35,100
it doesn't look much like it but this is
33
00:01:35,101 --> 00:01:37,680
the same style of processor that you
34
00:01:37,681 --> 00:01:40,200
might find in your phone arm processors
35
00:01:40,201 --> 00:01:42,289
have a lot of advantages first and
36
00:01:42,290 --> 00:01:43,789
foremost being that they're typically
37
00:01:43,790 --> 00:01:46,070
more power efficient thanks to their
38
00:01:46,071 --> 00:01:47,820
relatively lightweight and structure set
39
00:01:47,821 --> 00:01:51,539
so so much so that Nvidia claims these
40
00:01:51,540 --> 00:01:54,360
gray CPUs have twice the performance per
41
00:01:54,361 --> 00:01:57,420
watt of the latest x86 chips
42
00:01:57,421 --> 00:02:00,540
but the disadvantage is they also
43
00:02:00,541 --> 00:02:03,180
require software like your operating
44
00:02:03,181 --> 00:02:05,520
system and all the programs you need to
45
00:02:05,521 --> 00:02:07,450
run to be coded and compiled
46
00:02:07,451 --> 00:02:10,669
specifically for arm now for the PC
47
00:02:10,670 --> 00:02:12,950
market because 86 has been the standard
48
00:02:12,951 --> 00:02:15,660
for so long it's difficult to justify
49
00:02:15,661 --> 00:02:17,810
switching over to arm it would cost you
50
00:02:17,811 --> 00:02:19,310
so much in terms of backwards
51
00:02:19,311 --> 00:02:22,800
compatibility but in the data center the
52
00:02:22,801 --> 00:02:24,300
types of customers who are going to buy
53
00:02:24,301 --> 00:02:26,280
a processor like this are usually
54
00:02:26,281 --> 00:02:28,020
developing their own software anyway
55
00:02:28,021 --> 00:02:30,660
like let's say Google to run the
56
00:02:30,661 --> 00:02:32,579
algorithms that power Google search or
57
00:02:32,580 --> 00:02:34,980
YouTube recommendations for them
58
00:02:34,981 --> 00:02:37,680
switching over to arm isn't as big a
59
00:02:37,681 --> 00:02:40,140
deal and in fact companies like Amazon
60
00:02:40,141 --> 00:02:42,360
who are developing their own arm-based
61
00:02:42,361 --> 00:02:46,370
CPUs are already doing it and very
62
00:02:46,371 --> 00:02:49,800
effectively I mean hey if my next gaming
63
00:02:49,801 --> 00:02:52,620
CPU could be half the power draw and the
64
00:02:52,621 --> 00:02:54,720
same performance of my current one I'd
65
00:02:54,721 --> 00:02:57,120
be stoked but this is even better
66
00:02:57,121 --> 00:02:59,280
imagine if instead of one computer
67
00:02:59,281 --> 00:03:02,040
you're talking thousands or tens of
68
00:03:02,041 --> 00:03:04,800
thousands the savings start to become so
69
00:03:04,801 --> 00:03:07,010
large that it's less a question of can
70
00:03:07,011 --> 00:03:09,120
we afford this migration and more a
71
00:03:09,121 --> 00:03:12,300
question of can we afford not to make it
72
00:03:12,301 --> 00:03:14,400
now I didn't ask permission for this
73
00:03:14,401 --> 00:03:16,680
part but nobody seems to be stopping me
74
00:03:16,681 --> 00:03:19,370
or even really paying attention to me so
75
00:03:19,371 --> 00:03:22,920
let's take apart Grace Superchick
76
00:03:22,921 --> 00:03:27,590
on each gray Superchip is up to 480
77
00:03:27,591 --> 00:03:33,180
gigabytes of LP ddr5x ECC memory per CPU
78
00:03:33,181 --> 00:03:36,720
and what's really cool is that that can
79
00:03:36,721 --> 00:03:40,140
actually be accessed by either CPU over
80
00:03:40,141 --> 00:03:43,560
the nvlink interconnect that's how fast
81
00:03:43,561 --> 00:03:46,370
this new Envy link is the only downside
82
00:03:46,371 --> 00:03:48,480
to this approach since we're making
83
00:03:48,481 --> 00:03:51,180
comparisons to Apple is that just like
84
00:03:51,181 --> 00:03:54,000
with your M2 MacBook you better decide
85
00:03:54,001 --> 00:03:56,040
how much memory you want in your server
86
00:03:56,041 --> 00:03:58,260
right at the time you buy it unless you
87
00:03:58,261 --> 00:04:00,780
want to replace the entire compute
88
00:04:00,781 --> 00:04:02,580
engine while you perform a memory
89
00:04:02,581 --> 00:04:03,900
upgrade
90
00:04:03,901 --> 00:04:06,000
given that the rumored price of their
91
00:04:06,001 --> 00:04:09,230
h100 gpus is a hundred thousand dollars
92
00:04:09,231 --> 00:04:10,980
I don't even want to know what this
93
00:04:10,981 --> 00:04:13,140
thing costs but hopefully you get a bit
94
00:04:13,141 --> 00:04:15,180
of a discount when you buy it together
95
00:04:15,181 --> 00:04:18,540
with the gray Superchip CPU let me show
96
00:04:18,541 --> 00:04:19,500
you this
97
00:04:19,501 --> 00:04:21,000
can't believe they're letting me take
98
00:04:21,001 --> 00:04:23,240
this off the wall
99
00:04:23,241 --> 00:04:25,020
foreign
100
00:04:25,021 --> 00:04:26,820
okay
101
00:04:26,821 --> 00:04:29,580
success we have dropped nothing
102
00:04:29,581 --> 00:04:31,860
important so far today
103
00:04:31,861 --> 00:04:33,360
this
104
00:04:33,361 --> 00:04:37,080
is Grace Hopper on the one side we've
105
00:04:37,081 --> 00:04:40,380
got the same at 72 core Grace arm CPU
106
00:04:40,381 --> 00:04:43,380
that we just saw but on the other side
107
00:04:43,381 --> 00:04:48,780
the ooh shiny latest Nvidia h100 Hopper
108
00:04:48,781 --> 00:04:51,470
GPU you can probably see where this is
109
00:04:51,471 --> 00:04:54,720
going just like with the Dual CPU Grace
110
00:04:54,721 --> 00:04:58,130
module these two are also Envy link chip
111
00:04:58,131 --> 00:05:00,470
to chip interconnected meaning that the
112
00:05:00,471 --> 00:05:04,100
CPU and GPU have a whopping 900
113
00:05:04,101 --> 00:05:07,190
gigabytes per second of theoretical
114
00:05:07,191 --> 00:05:09,720
bandwidth to talk to each other so for
115
00:05:09,721 --> 00:05:13,440
some perspective a GPU using a full 16
116
00:05:13,441 --> 00:05:16,800
Lane Gen 5 pcie slot would only have
117
00:05:16,801 --> 00:05:19,500
about 64 gigabytes a second of peak
118
00:05:19,501 --> 00:05:22,910
throughput that is 1 14 as much as this
119
00:05:22,911 --> 00:05:24,840
and that's far from the only
120
00:05:24,841 --> 00:05:26,940
mind-bending number that this thing is
121
00:05:26,941 --> 00:05:29,940
capable of while the CPU side uses the
122
00:05:29,941 --> 00:05:33,720
same up to 480 gigabytes of lpddr5x
123
00:05:33,721 --> 00:05:36,970
for the GPU side they need much faster
124
00:05:36,971 --> 00:05:40,820
hbm3 memory that runs at a whopping four
125
00:05:40,821 --> 00:05:43,560
terabytes per second
126
00:05:43,561 --> 00:05:46,190
it's about four times faster that's why
127
00:05:46,191 --> 00:05:47,880
the memory needs to be right on the
128
00:05:47,881 --> 00:05:50,820
package right next to the GPU
129
00:05:50,821 --> 00:05:53,880
now all that is great and cool and all
130
00:05:53,881 --> 00:05:57,120
but hbm is very expensive and as you can
131
00:05:57,121 --> 00:05:59,220
see there's only so much space here so
132
00:05:59,221 --> 00:06:03,600
the h100 only gets 96 gigabytes of
133
00:06:03,601 --> 00:06:06,410
memory okay yeah for gaming that
134
00:06:06,411 --> 00:06:09,120
certainly sounds like a lot but AI data
135
00:06:09,121 --> 00:06:11,580
sets can involve terabytes of data so it
136
00:06:11,581 --> 00:06:13,800
can get used up very quickly that's
137
00:06:13,801 --> 00:06:15,900
where the interconnect comes in it
138
00:06:15,901 --> 00:06:19,020
allows the GPU to access the cpu's
139
00:06:19,021 --> 00:06:21,780
memory in a very direct and transparent
140
00:06:21,781 --> 00:06:24,960
way giving the h100 hopper GPU a
141
00:06:24,961 --> 00:06:27,900
functional memory capacity of nearly 600
142
00:06:27,901 --> 00:06:31,310
gigabytes in Practical terms according
143
00:06:31,311 --> 00:06:33,530
to Nvidia that puts Grace Hopper
144
00:06:33,531 --> 00:06:35,340
anywhere from about two and a half times
145
00:06:35,341 --> 00:06:39,530
to nearly four times as fast as an x86
146
00:06:39,531 --> 00:06:41,960
CPU paired with their last generation
147
00:06:41,961 --> 00:06:45,960
a100 GP and where things get really wild
148
00:06:45,961 --> 00:06:48,470
is in the data center with an Envy link
149
00:06:48,471 --> 00:06:51,680
switch system you could connect up to
150
00:06:51,681 --> 00:06:55,310
256 gpus together giving them access to
151
00:06:55,311 --> 00:06:58,860
up to 150 terabytes of high bandwidth
152
00:06:58,861 --> 00:07:00,530
memory I mean you guys remember that
153
00:07:00,531 --> 00:07:02,750
crazy Mars Lander demo that we showed
154
00:07:02,751 --> 00:07:05,030
off on the petabyte of flash array you
155
00:07:05,031 --> 00:07:07,680
could load that entire 1 billion Point
156
00:07:07,681 --> 00:07:10,020
data set into memory in that
157
00:07:10,021 --> 00:07:12,410
configuration and still have 50
158
00:07:12,411 --> 00:07:16,080
terabytes to spare now this module
159
00:07:16,081 --> 00:07:18,410
get more power hungry than the Dual CPU
160
00:07:18,411 --> 00:07:21,840
version a thousand versus 500 watts per
161
00:07:21,841 --> 00:07:26,160
module but I mean that's for CPU GPU and
162
00:07:26,161 --> 00:07:28,620
RAM for both of them and with this kind
163
00:07:28,621 --> 00:07:30,300
of performance
164
00:07:30,301 --> 00:07:32,810
of course not everybody wants to move to
165
00:07:32,811 --> 00:07:35,580
an arm hybrid CPU GPU architecture so
166
00:07:35,581 --> 00:07:37,909
Nvidia is still going to be supporting
167
00:07:37,910 --> 00:07:41,280
their uh old-fashioned configurations be
168
00:07:41,281 --> 00:07:45,180
they h100 gpus and a pcie form factor or
169
00:07:45,181 --> 00:07:49,680
their hgx h100 with up to eight smx-5
170
00:07:49,681 --> 00:07:53,880
gpus each of these draws a massive 700
171
00:07:53,881 --> 00:07:57,120
Watts making an RTX 4090 look like a
172
00:07:57,121 --> 00:07:59,810
child's play thing and supports Envy
173
00:07:59,811 --> 00:08:03,410
link between these gpus and NV switch to
174
00:08:03,411 --> 00:08:06,620
additional servers this is the
175
00:08:06,621 --> 00:08:09,530
g593-sd 0 and gigabyte was very proud of
176
00:08:09,531 --> 00:08:12,000
the fact that they are the first Nvidia
177
00:08:12,001 --> 00:08:16,250
certified HDX h100 8 GPU server in a 5u
178
00:08:16,251 --> 00:08:18,660
chassis man that is a lot of compute in
179
00:08:18,661 --> 00:08:20,870
a tiny space Jake's in my ear here
180
00:08:20,871 --> 00:08:22,020
telling me I should pull one of the
181
00:08:22,021 --> 00:08:23,750
power supplies but if you've noticed it
182
00:08:23,751 --> 00:08:25,190
getting darker it's because they're
183
00:08:25,191 --> 00:08:27,530
actually shutting down the pre-show and
184
00:08:27,531 --> 00:08:28,860
they're trying to get us out of here but
185
00:08:28,861 --> 00:08:30,000
there is one more thing that we wanted
186
00:08:30,001 --> 00:08:32,580
to talk about where'd it go dang it Jake
187
00:08:32,581 --> 00:08:34,970
no oh my God oh my God
188
00:08:34,971 --> 00:08:37,970
okay well this is uh no wait this isn't
189
00:08:37,971 --> 00:08:39,710
the one I wanted okay it's a connect X7
190
00:08:39,711 --> 00:08:41,940
this is an even faster network card so
191
00:08:41,941 --> 00:08:44,640
this is probably the first Nvidia
192
00:08:44,641 --> 00:08:46,980
developed melanox network card given
193
00:08:46,981 --> 00:08:48,660
that uh the acquisition was what about
194
00:08:48,661 --> 00:08:50,700
two years ago six six
195
00:08:50,701 --> 00:08:53,880
yeah but Nvidia didn't buy melanox just
196
00:08:53,881 --> 00:08:56,600
to make faster connectex cards
197
00:08:56,601 --> 00:09:00,950
no it was to make these this is a blue
198
00:09:00,951 --> 00:09:03,830
field three so it has networking on it
199
00:09:03,831 --> 00:09:05,880
this is a 100 gigabit one but it's
200
00:09:05,881 --> 00:09:08,220
available it speeds up to 400 gigabit
201
00:09:08,221 --> 00:09:10,560
but what's really special about it is
202
00:09:10,561 --> 00:09:13,800
that it has up to 16 processing cores on
203
00:09:13,801 --> 00:09:16,980
it why you might ask well just like in
204
00:09:16,981 --> 00:09:19,450
the old days when we started offloading
205
00:09:19,451 --> 00:09:22,200
tcpip processing to our network cards
206
00:09:22,201 --> 00:09:24,380
rather than having our CPU handle them
207
00:09:24,381 --> 00:09:27,060
this is going to offload all kinds of
208
00:09:27,061 --> 00:09:29,510
interesting things like encryption of
209
00:09:29,511 --> 00:09:31,800
your network traffic or say for example
210
00:09:31,801 --> 00:09:33,950
handling managing your file system
211
00:09:33,951 --> 00:09:36,360
because when you're someone like an AWS
212
00:09:36,361 --> 00:09:38,220
and you want to squeeze as much revenue
213
00:09:38,221 --> 00:09:40,740
as possible out of every CPU in your
214
00:09:40,741 --> 00:09:42,360
data center you don't want it handling
215
00:09:42,361 --> 00:09:45,060
stupid BS that you could just offload to
216
00:09:45,061 --> 00:09:48,120
your network card so the idea here is to
217
00:09:48,121 --> 00:09:50,270
free up CPU resources that can be leased
218
00:09:50,271 --> 00:09:53,160
to customers by putting them onto the
219
00:09:53,161 --> 00:09:55,140
network card itself and this is
220
00:09:55,141 --> 00:09:57,000
especially true for software where the
221
00:09:57,001 --> 00:10:00,360
developer sells you a license per core
222
00:10:00,361 --> 00:10:01,860
that's why even though these are going
223
00:10:01,861 --> 00:10:04,080
to be wildly expensive
224
00:10:04,081 --> 00:10:07,320
a lot more than the 4060 TI Nvidia is
225
00:10:07,321 --> 00:10:09,890
going to sell shed loads of them just
226
00:10:09,891 --> 00:10:12,560
like I sold this Segway to our sponsor
227
00:10:12,561 --> 00:10:15,180
pulseway are you sick of feeling like a
228
00:10:15,181 --> 00:10:17,040
prisoner changed to a desk managing it
229
00:10:17,041 --> 00:10:19,800
systems Unleash Your Inner it hero with
230
00:10:19,801 --> 00:10:21,300
pulse waves remote monitoring and
231
00:10:21,301 --> 00:10:23,390
management software pulse waste platform
232
00:10:23,391 --> 00:10:25,010
gives you the power to manage your it
233
00:10:25,011 --> 00:10:27,180
infrastructure from anywhere even from
234
00:10:27,181 --> 00:10:29,100
the comfort of your own couch and with
235
00:10:29,101 --> 00:10:30,950
real-time alerts and notifications you
236
00:10:30,951 --> 00:10:32,700
can be the first to know about potential
237
00:10:32,701 --> 00:10:34,620
issues before anyone else on your team
238
00:10:34,621 --> 00:10:36,600
it's accessible through whatever devices
239
00:10:36,601 --> 00:10:38,220
close to you thanks to their convenient
240
00:10:38,221 --> 00:10:39,950
apps allowing you to control your it
241
00:10:39,951 --> 00:10:42,060
systems like a boss even if you're
242
00:10:42,061 --> 00:10:44,330
lounging in your pjs so say goodbye to
243
00:10:44,331 --> 00:10:46,140
the boring routine of it management and
244
00:10:46,141 --> 00:10:48,000
hello to the fun of being an I.T hero
245
00:10:48,001 --> 00:10:50,160
with pulseways advanced technology don't
246
00:10:50,161 --> 00:10:51,770
wait this is your chance to become a
247
00:10:51,771 --> 00:10:53,820
legend in the IT world just try pulse
248
00:10:53,821 --> 00:10:55,560
wave for free today and experience the
249
00:10:55,561 --> 00:10:57,360
power of simplified it infrastructure
250
00:10:57,361 --> 00:10:59,270
management click the link below to get
251
00:10:59,271 --> 00:11:01,500
started if you guys enjoyed this video
252
00:11:01,501 --> 00:11:04,070
why don't you check out oh the petabyte
253
00:11:04,071 --> 00:11:05,040
gosh
254
00:11:05,041 --> 00:11:06,300
this is a good one well we're at the
255
00:11:06,301 --> 00:11:08,640
gigabyte Booth come on uh the g-rad one
256
00:11:08,641 --> 00:11:12,480
yeah actually no new one new new
257
00:11:12,481 --> 00:11:13,980
one a three
258
00:11:13,981 --> 00:11:17,720
One X four I mean damn it
18320
Can't find what you're looking for?
Get subtitles in any language from opensubtitles.com, and translate them here.