-
2 weeks ago | 
semianalysis.com | Dylan Patel |Daniel Nishball |Ivan Chiam
It has been long claimed that AMD’s AI servers can achieve better inference performance per total cost of ownership (TCO) than Nvidia. We have spent the past six months investigating and validating this claim through a comprehensive analysis and benchmarking of inference solutions offered by both Nvidia and AMD. We expected to arrive at a simple answer, but instead the results were far more nuanced and surprising to us.
-
1 month ago | 
semianalysis.com | Dylan Patel |Daniel Nishball |Myron Xie |Wega Chu
Huawei is making waves with its new AI accelerator and rack scale architecture. Meet China’s newest and most powerful Chinese domestic solution, the CloudMatrix 384 built using the Ascend 910C. This solution competes directly with the GB200 NVL72, and in some metrics is more advanced than Nvidia’s rack scale solution. The engineering advantage is at the system level not just at the chip level, with innovation at the accelerator, networking, optics, and software layers.
-
Feb 13, 2025 | 
semianalysis.com | Dylan Patel |Jeremie Eliahou Ontiveros |Daniel Nishball |Reyk Knuhtsen
Cluster deployments are an order of magnitude larger in scale with Gigawatt-scale datacenters coming online at full capacity much faster than most believe. As such, there are considerable design changes that Datacenter developers planning future sites must consider. We previously covered the Electrical system of Datacenters and how the rise of Generative AI is impacting datacenter design and equipment suppliers.
-
Jan 15, 2025 | 
semianalysis.com | Dylan Patel |Daniel Nishball |Myron Xie |AJ Kourabi
The US government lobbed the largest salvo in the new technology cold war with its new Framework for Artificial Intelligence Diffusion. These new export restrictions are completely unprecedented in scope and scale, with many calling the efforts overzealous or misguided. The regulation at its core is targeted at preventing China from accessing AI compute to build frontier models.
-
Dec 10, 2024 | 
semianalysis.com | Dylan Patel |Daniel Nishball |AJ Kourabi
There has been an increasing amount of fear, uncertainty and doubt (FUD) regarding AI Scaling laws. A cavalcade of part-time AI industry prognosticators have latched on to any bearish narrative they can find, declaring the end of scaling laws that have driven the rapid improvement in Large Language Model (LLM) capabilities in the last few years.
-
Dec 3, 2024 | 
semianalysis.com | Dylan Patel |Daniel Nishball |Reyk Knuhtsen
Amazon is currently conducting one of the largest build-out of AI clusters globally, deploying a considerable number of Hopper and Blackwell GPUs. In addition to a massive Capex invested into Nvidia based clusters, AWS is also investing many billions’ dollars worth of capex into Trainium2 AI clusters. AWS is currently deploying a cluster with 200k+ Trainium2 chips for Anthropic called “Project Rainer”.
-
Nov 20, 2024 | 
semianalysis.com | Dylan Patel |Daniel Nishball |Myron Xie |Wega Chu
Nvidia’s Blackwell family is encountering major issues in reaching high volume production. This setback has impacted their production targets for Q3/Q4 2024 as well as the first half of next year. This affects Nvidia’s volume and revenue, as we detailed in the Accelerator Model on July 22nd. In short Nvidia’s Hopper is extended in lifespan and shipments to make up for a chunk of the delays.
-
Nov 20, 2024 | 
semianalysis.com | Dylan Patel |Daniel Nishball
The rise of the AI Neoclouds has captivated the attention of the entire computing industry. Everyone is using them for access to GPU compute, from enterprises to startups. Even Microsoft is spending ~$200 million a month on GPU compute through AI Neoclouds despite having their own datacenter construction and operation teams. Nvidia has heralded the rapid growth of several AI Neoclouds through direct investments, large allocations of their GPUs, and accolades in various speeches and events.
-
Dec 24, 2023 | 
exponentialview.co | Azeem Azhar |Dylan Patel |Daniel Nishball |Nathan Lambert
Hi all, here with our last regular Sunday edition of the year as we slow down to rest & recharge. I want to wish you all a joyful holiday season ā and a happy Christmas if you celebrate on the 25th!P.S. If you are in a pinch and need a last-last-minute gift for a future-curious relative, may I suggest a year of Exponential View membership?
-
Aug 30, 2023 | 
semianalysis.com | Dylan Patel |Daniel Nishball |Myron Xie
Endless waffling has been done regarding competition for Nvidia’s dominance in AI, and potential for competitors. While much of this discussion has centered around AMD, with some sidebars on Intel and Cerebras, the fact is that in 2023, no one besides Nvidia will generate even $1B revenue from chips that run large language models. Scratch that, there is one other player.
