Locator: 48432TECH.
Beth is recycling some of her stuff but it's still valuable.
Link here.
From what little I know, there's nothing "new" in this article, but it says what "we all" already know. But just a lot "more of it."
Link here.
Big Tech is spending tens of billions quarterly on AI accelerators, which has led to an exponential increase in power consumption. The rise of generative AI and surging GPU shipments is causing data centers to scale from tens of thousands to 100,000-plus accelerators, shifting the emphasis to power as a mission-critical problem to solve.
As Nvidia, AMD, and soon Intel begin to roll out their next generation of AI accelerators, the focus is now shifting towards power consumption per chip, whereas the focus has been primarily on compute and memory. As each new generation boosts computing performance, it also consumes more power than its predecessor, meaning that as shipment volumes rise, so does total power demand.
Nvidia’s A100 max power consumption is 250W with PCIe and 400W with SXM (Server PCIe Express Module), and the H100’s power consumption is up to 75% higher versus the A100. With PCIe, the H100 consumes 300-350W, and with SXM, up to 700W. The 75% increase in GPU power consumption happened rapidly, within two brief years, across one generation of GPUs.
When we look at other GPUs on the market today, AMD’s MI250 accelerators draw 500W of power, up to 560W at peak, while the MI300x consumes 750W at peak, up to a 50% increase. Intel’s Gaudi 2 accelerator consumes 600W, and its successor, the Gaudi 3, consumes 900W, again another 50% increase over the previous generation. Intel’s upcoming hybrid AI processor, codenamed Falcon Shores, is expected to consume a whopping 1,500W of power per chip, the highest on the market.
Nvidia’s upcoming Blackwell generation boosts power consumption even further, with the B200 consuming up to 1,200W, and the GB200 (which combines two B200 GPUs and one Grace CPU) expected to consume 2,700W. This represents up to a 300% increase in power consumption across one generation of GPUs with AI systems increasing power consumption at a higher rate. SXM allows the GPUs to operate beyond the PCIe bus restrictions, offer higher memory bandwidth, high data throughput and higher speeds for maximal HPC and AI performance, thus drawing more power.
It’s important to note that each subsequent generation is likely to be more power-efficient than the last generation, such as the H100 reportedly boasting 3x better performance-per-watt than the A100, meaning it can deliver more TFLOPS per watt and complete more work for the same power consumption. However, GPUs are becoming more powerful in order to support trillion-plus large language models.
From Big Tech’s perspective, we’re still in the early stages of this AI capex cycle. Most recently, we covered how Big Tech is boosting capex by more than 35% YoY in 2024, likely upwards of $200 billion to $210 billion, predominantly for AI infrastructure. The majority is flowing to GPU purchases and custom silicon, to power AI training, model development, and to meet elevated demand in the cloud.
2023 was a breakout year for Nvidia’s data center GPUs, with reports placing annual shipments at 3.76 million, for an increase of more than 1.1 million units YoY. A report stated that at peak of 700W and ~61% annual utilization, each GPU would draw 3.74 MWh; this means that Nvidia’s 3.76 million GPU shipments could consume as much 14,384 GWh (14.38 TWh). A separate report estimated that with 3.5 million H100 shipments through 2023 and 2024, that H100 alone could see total power consumption of 13.1 TWh annually.
The 14.4 TWh is equivalent to the annual power needs of more than 1.3 million households in the US. This also does not include AMD, Intel, or any of Big Tech’s custom silicon, nor does it take into account existing GPUs deployed or upcoming Blackwell shipments in 2024 and 2025. As such, the total energy consumption is likely to be far higher by the end of the year as Nvidia’s Blackwell generation comes online in larger quantities.
Also from Forbes. But much more recent, March 26, 2025. Link here.
Unfortunately the article doesn't say much / doesn't add much.
TFLOPs per watt? Link here.
Most complicated human endeavor on earth? The global passenger airline sector.
AI says otherwise.
Link here.
Change this to most complicated free market endeavor and AI gives us something else.
Link here.
Change this to most complicated free market industrial sector and AI gives us something else.
The weather is but a small piece of global passenger airline sector.
Back to TFLOPs per watt.
Wiki:
In computing, performance per watt is a measure of the energy efficiency of a particular computer architecture or computer hardware. Literally, it measures the rate of computation that can be delivered by a computer for every watt of power consumed.
Performance per watt has been suggested to be a more sustainable measure of computing than Moore's Law.
System designers building parallel computers, such as Google's hardware, pick CPUs based on their performance per watt of power, because the cost of powering the CPU outweighs the cost of the CPU itself.
Spaceflight computers have hard limits on the maximum power available and also have hard requirements on minimum real-time performance. A ratio of processing speed to required electrical power is more useful than raw processing speed.
********************************
Word For The Day: Deictic
Pronunciation: dike-tik, emphasis on the first syllable
Can be used as an adjective and a noun:
relating to or denoting a word or expression whose meaning is dependent on the context in which it is used (such as here, you, me, that one there, or next Tuesday).
For example, "you" could refer to the one person to whom you are speaking or to the "entire audience" of a thousand.
A "deictic shift" is when one shifts from "you" in the singular to "you" as a group in the same piece of writing.
It's one of those words you will never see or never use but when you do you will wonder why it's taken so long.
