State of AI Report: Compute Index V3 update!
Today, we release V3 of the State of AI Report Compute Index in collaboration with the team at Zeta Alpha.
You'll now find updated counts of AI research papers using chips from NVIDIA, TPUs, ASICs, FPGAs, and AI semi startups, as well as updates to H100 cluster sizes. You heard it here first 😉
Meta leaps with gargantuan H100 numbers
Last October, we published a chart featuring disclosed H100 clusters at big tech companies, AI labs and AI-first companies ranging from Google to DeepL. Just a few days ago, Mark Zuckerberg stated the sheer scale of Meta’s AI infrastructure on their earnings call:
The first is world-class compute infrastructure. I recently shared that by the end of this year we'll have about 350k H100s and including other GPUs that’ll be around 600k H100 equivalents of compute. We're well-positioned now because of the lessons that we learned from Reels. We initially under-built our GPU clusters for Reels, and when we were going through that I decided that we should build enough capacity to support both Reels and another Reels-sized AI service that we expected to emerge so we wouldn't be in that situation again. And at the time the decision was somewhat controversial and we faced a lot of questions about capex spending, but I'm really glad that we did this. (Source)
This makes Meta likely one of the largest GPU cluster operators in the world, further expanding their AI Research SuperCluster investments from 2 years ago.
NVIDIA’s lead on all other AI chip players expands
We’ve updated the AI paper count per chip with final 2023 numbers and provide an early extrapolation of 2024 using numbers from January 2023.
You’ll notice that in 2023 there were over 29k AI research papers that made use of any type of NVIDIA hardware. By contrast, all FPGAs sum to 823 papers, Google’s TPU comes in at 167, and the sum of the 5 AI startups comes to 243 papers. Based on numbers so far in 2024, it looks like Google’s TPU could be set for a big year of relative growth.
This is an enormous gap (note the log scale).
NVIDIA’s A100 overtakes the V100 as its most popular chip for AI research
In this graph, we show you NVIDIA specific data. Last October, we noted that the most popular chip in AI research was the V100 (est. 2017), a card that is several years old already. You can see now in updated final 2023 data that the A100 has overtaken the V100 in AI research papers. Meanwhile, the RTX 3090 (est. 2020) continues to rise in popularity.
Contrary to all the social and media chatter about the H100, this highest performance chip is used in only 0.55% of all AI research papers (160 in total). The growth potential for its usage is enormous.
Cerebras overtook its peer AI semiconductor startups in 2023
While overall counts are very low amongst AI semiconductor startups in AI research, the most usage in papers now comes from Cerebras. Following them is Graphcore, Cambricon and SambaNova. This seems to corroborate with the open source and developer community efforts that Cerebras have invested since the rise of LLMs, so kudos! If early 2024 numbers are extrapolated out to the year end, this trend looks to continue.
A few notes
- We take the view that usage of chips in AI research papers (early adopters) is a leading indicator of industry usage.
- Papers using AI semi startup chips almost all have authors from the startup.
See the live charts here: www.stateof.ai/compute