The Good, The Bad and the Ugly
Was listening to a podcast the other day and the subject was semiconductors which led to AI which led to Data Centers. Admit it you haven’t thought much about data centers as a limiting factor in our move to the AI age.
The podcast was with Jonathan Ross the Founder & CEO of Groq, the creator of the world’s first Language Processing Unit (LPUTM). Great interview as usual by Harry Stebbings of 20 Minute VC
Types of Data Centers
No there is not only one type of Data Center, but according to most there are 5 common Data Center types, let’s just quickly list them out. All descriptions are from https://www.datacenterfrontier.com/
- Enterprise | “An enterprise data center is a private data center facility that supports a single organization.”
- MTDC also know as Colo or Colocation | “Multi-tenant data centers (also known as colocation data centers) offer data center space to businesses that want to host their computing hardware and servers offsite.”
- Hyperscale | “Hyperscale data centers are designed to support very large-scale IT infrastructure.” Worth noting “Interesting fact: Amazon, Microsoft and Google account for more than half of all hyperscale data centers.”
- Edge or Micro | “These types of data centers are small and located near the people they serve to handle real-time data processing, analysis and action, making low-latency communication with smart devices possible.”
- Container or Modular | “A container data center is usually a module or shipping container that’s packaged with ready-made, plug-and-play data center components: servers, storage, networking gear, UPS, generators, air conditioners, etc.”
What could possibly be controversial about a Data Center, it is really a huge building with computers to run software in the cloud. Oh, that’s right they use huge amounts of electricity.
Why would Data Centers suddenly need more power? According to Goldman Sachs:
“On average, a ChatGPT query needs nearly 10 times as much electricity to process as a Google search.”
Guess how I found that nugget, yup ChatGPT. So that means I just burned a lot of watts to find out that AI drives Data Center Energy usage. That is either ironic or full-circle.
Unless you are an expert in electrical power grids most of the numbers you find about Data Centers will be meaningless to confusing. I like this chart from Goldman Sachs that shows the relative power usage by Data Centers. The biggest driver in the US’s power demand CAGR from 2022 to 2030 is by far Data Centers with transportationa and others lagging behind. So Data Centers are a newer phenomenum than manufacturing but it is where the growth is going.
Rather than simply say data centers use lots of power, let’s double click on this one. From Berkeley Lab “2024 United States Data Center Energy Usage Report” this chart shows where the power goes. One recurring theme on this blog for data centers, nothing is as simple as it seems.
It is not just Data Centers = Power Usage
Notice the water consumption boxes on the right, that too is an issue as we see data centers spring up in the desert as there is cheap land. Now that we see a breakdown of the energy usage inside a data center it makes you think…data center designers need to be equally expert at energy usage and data center layout.
Power Grids
Electricity often seems ubiquitous if you live in a Western country but if you live in a war zone like Ukraine today or others you likely do not take the power grid for granted. I live in a nice area of Sonoma County but this is also considered Fire Country so we get occasional brown outs for safety reasons.
A fascinating bit from the Goldman Sachs report on Data Centers is the average age of a power grid. I don’t think power grids need to be updated annually or even often like a bank’s software does, but you also you can not ignore the age or modern nature of a power grid.
First thing we notice is that China has the youngest power grid of the bunch. I am unclear if Eastern Europe would have a newer or older power grid than the rest of Europe. But Europe’s average power grid age of 50 years does not bode well for the continent as we ramp up for the age of AI.
Overall you will notice the up and coming economies of the Middle East and SE Asia are next newest power grids after China. Again by country there could be vast differences between Brazil, Argentina and Peru in Latin America.
The issue is:
“Europe’s power demand could grow by 40% and perhaps even 50%, according to Goldman Sachs Research.”
Time to invest in national power grids. As you may have guessed, that is an expensive proposition and likely not politically popular as global governments are moving towards one of two positions:
“No more taxes, cut spending”
OR
“People first, data second”
Well neither of these positions bodes well for a big government led power grid funding push. But we also know AI will not be denied and even the European’s attempts to over-regulate AI will be met headfirst by progress.
Goldman Sachs estimates: “€800 billion ($861 billion) in spending on transmission and distribution over the coming decade.” That money is likely to come from the governments of Europe. Maybe paid by taxes on individuals or by corporates but the money needs to be spent.
Back to Data Centers
The power grid like water or clean air are presumed to be utilities delivered by fully functioning governments.
Looking at the data center data, we see a weird decrease in the number of data centers from 2015 to today. Well not really it was a shift from companies having an IT closet housing a server being counted as a data center or a bank hosting an HP-3000 mini computer that shifted to an off-site colo server farm inside of an AWS data center. It is more about the coming of actual data centers over IT servers dispersed throughout a company or government building.
Notice how Asia Pacific is outpacing the US for hyperscale and colo data centers.
- Cheaper land
- Cheaper labor
- Cheaper power
But are they sustainable? Every country needs to do better with energy efficiency and it is likely they are all working on this very issue including as noted above by modernizing their national power grid.
Get Dion F. Lisle’s stories in your inbox
Join Medium for free to get updates from this writer.
Just imagine expecting your 1990's Chevrolet Suburban to get the same mileage as a 2025 Hybrid Toyota SUV. Clearly the newer one will be better.
In 2020 what would a Data Center spec look like
- “Small data centers ranged from 5,000 to 20,000 square feet, hosting 500 to 2,000 servers, with power demands between 1 to 5 megawatts (MW). “
- “Larger facilities, spanning 100,000 square feet or more, accommodated tens of thousands of servers, requiring 20 MW to over 100 MW.”
Looking forward to 2030 which is actually closer to us today than 2020.
- “Data centers are projected to continue expanding, with facilities exceeding 20 million square feet and power capacities surpassing 10 GW to meet escalating demands from AI and cloud computing.”
Data center’s physical footprint is going from 100,000 square feet to 20 million square feet
Hence my point about cheap land from above. But do not under-estimate the USA’s vast supply of cheap land. Not all of America is NYC or Los Angeles where land can cost $3.9 million per acre according to Landsearch.
As you will see from this list of the largest data centers in the world, the US and China dominate the list.
The time it takes to build a modern data center has increased from 1–2 years to 3–4 years. source: CBRE. It has to be noted a lot of the increase in building timeline is from supply chain issues of availability of chips, servers and yes access to power.
Texas …. Land of the Data Center
Yes, Texas of all the places in the USA is gunning (pun intended) to become the Data Center capital of the free world.
“A Houston-based company is planning to construct a massive 50,000-acre data center hub near Laredo, Texas, dubbed “Data City, Texas.” — CBRE Having lived in Texas years ago I can attest they tend to do thing BIGGER. Well it turns out their focus on data centers is no exception.
But actually Texas lags Northern Virginia by quite a bit. There is a simple explanation for this as Virginia is near to Washington DC and noting the US Government is a big user of data centers. You may have noticed in the Largest Data Centers in the world list above that the list includes two Virginia locations and no Texas locations.
Texas as the center of US data centers is an aspirational position more than a current one.
If you are like me you picture a data center as a large air conditioned non-descript building in the middle of nowhere. Fair enough, but have you thought about the many layers of tech that resides under that facade?
Well here you go; a Datacenter Infrastructure Reference Architecture.
I am guessing it is more complexity than you thought. Like the power grid discussion above, the actual data center architecture needs a lot of explaining, more than we will cover here today. The key point of this eye chart diagram is that there is more to a data center than meets the eye.
With that said, I would not want to be responsible for designing or building this complex beast.
Summary
Hopefully you have a better idea of the growth in the Data Center market and the underlying complexity. So next time someone suggests that a new AI model just needs more data center capacity, maybe show them this blog and mention that data centers take 3+ years to build.
You know your local freeway that is 3 lanes and always backed up, ok if you live in Los Angeles make that your 5 lane freeway. But the point is by the time the local government gets funding for and plans for and builds a new lane costing $1M per mile or more, what do you see….more traffic.
Why is that? Because adding freeway lanes takes 5+ years while our population and number of cars increases all the time at double digit growth rates.
How do you get ahead of this issue? Building more capacity than you need. But that is expensive and time consuming. No private company wants to sit on excess capacity, yet that is what is needed as AI’s data center needs have already exceeded our expectations.
The other key to solving this issue is efficency. Look at the awful picture of the 5 north and 110 north above in LA. (I have driven this path and it sucks) Now instead of all those cars with mostly single drivers in them, imagine a train that runs the same route with thousands of riders or multiple buses with 50 riders instead of 50 cars.
That is the same for data centers. We don’t just need to build bigger data centers we need self-sustaining highly energy efficient data centers.
As we move forward to an uncertain but exciting AI future, let’s be as smart about our data centers and their energy needs as we are about accessing all human knowledge every printed.
Wine and Data Centers
I bet you thought I could not correlate data centers and wine.
Cooling.
Data centers need to be cool, dark places to ensure the servers do not overheat and stop. Wine cellars need to be cool, dark places to ensure your wine does not spoil.
As data centers use tremendous amounts of energy and also water. Yes many data centers use water to cool the servers and keep the temperature low. That makes sense. Unless of course you are in Texas where there is a never ending shortage of water for farming and fighting wild fires.
Should data centers be built underground like wine cellars?
Should your next data center be in a cave?
If that would reduce the need for electricity and water for cooling at a nominal additional cost the answer would be yes. Also a huge bonus in Texas would be protection from Tornados with the data center underground.
Maybe my next business will be a Sonoma Wine and Data Center cave with tours of the data center while sipping a lovely Pinot.
Imagine the beautiful Buena Vista Wine cave below but with servers on one side and wine barrels on the other or better yet, servers inside of wine barrels.
