What do we mean by compute scarcity?

Question

Been discussing this in my network, and it seems everyone has a different interpretation of what we mean by "compute scarcity". Some have pointed out that GPUs are in fact heavily under-utilized and just sitting idle.My understanding previously was that it was power/electricity issue (at least in the states). Hence why ppl in the countryside are disillusioned with the big tech since power is diverted to data centers to feed GPUs and cooling systems and they are either having power outages or their bills are going up (or both). Is that true though?You also need real estate - you can't just build a data center anywhere. Ignoring the cooling issue for a second, it could be the reason why Google, SpaceX and others are considering orbiting data centers.What's your take? Does anyone have more info? Is it actual GPU/silicon scarcity or power limitations or something else?

ethigent · Accepted Answer

I would say it is the "metrics driven environment" surrounding Datacenter operations. Which is already sliced and diced to conform to Rack U-age and Cage space. So customers within said Datacener can be charged or each element. Look for water to become a true metric as well!That said all of this primarily rolls up into token usage and AI. Since use of AI resources is based on token usage, each element can also be sliced and diced down to the token cost. A very fair metric frankly, but contributing to compute scarcity as an overall concept.

tim-tday · Answer

Stop saying orbital data centers. They will be compute constellations or nothing at all. (Data center is a warehouse with servers in it. That’s impossible in space with current technology). But a million satellites with a gpu? Yep.
Terrestrial Data centers need power, cooling, zoning approval, data connectivity. (Google built big data centers in areas either cheap power on dark fiber lines —customer facing infrastructure needs to be near population centers for latency reasons)
The compute scarcity is projected. Graph the demand growth over time alongside the current data center build timelines and demand will outstrip supply at some point on the order of months or tens of months depending on the assumptions you make in your graph. This makes demand predictions rather important.
The objection normally people have to data centers isn’t that “power gets diverted”. It’s that more power demand increases pressure on limited supply (power companies respond by deploying less cost-effective generation technologies and your household power bill goes up.) oh and they’re loud and they use a lot of water for evaporative cooling.
My take is that it will neither be as bad as the worst case scenario nor as good as the best case.
And power bills will go up for everyone so deploy rooftop solar ASAP. and ram prices will be elevated for at least six months. and debt markets will become exhausted so not all the data centers in the planning phase will be able to find funding. And maybe when that happens the music stops and everyone starts looking for a chair. And maybe there aren’t enough chairs for everyone. But I’m not an economist so I don’t really know how that plays out.