Part 2 - Data Center Prediction for 2024: GPU Accounts for ~65% of BOM, Significant Demand for Renewable Energy & PUE Closer to 1
In January 2023, McKinsey had projected the US data center demand would grow ~10% a year through 2030. However, the actual growth we're witnessing in 2023 and this year is markedly surpassing these projections. In Part 1 of the series, we looked at how GPUs consume 3-4x more energy, reviewed projected data center capacity and colo providers. Our recent dialogues with clients have increasingly revolved around the nuanced differences in the BOM between CPU and GPU configurations and the critical takeaways from these distinctions.
Given GPU accounts for 65% of Server BOM, clients are seeking effective strategies to enhance the utilization rates of GPUs within data centers. The high cost of GPUs necessitates their optimal use to justify the expenditure. Our analysis indicates that during AI model training phases, GPUs tend to reach utilization peaks of 80-90%, demonstrating their efficiency in high-demand scenarios. However, for inference tasks, the utilization drops to a moderate 50-60%.
In this part 2 of series, we will review:
Greenfield Data Center Cost Breakdown - Traditional vs. AI
BOM % comparison between CPU and GPU Data Centers Racks $
GPU utilization a critical factor for clients
Investments in cooling
Renewable Energy Sector Getting a Lot of Attention
PUE closer to 1.0 is possible
Players
Please note: The insights presented in this article are derived from confidential consultations our team has conducted with clients across private equity, hedge funds, startups, and investment banks, facilitated through specialized expert networks. Due to our agreements with these networks, we cannot reveal specific names from these discussions. Therefore, we offer a summarized version of these insights, ensuring valuable content while upholding our confidentiality commitments.