What do we mean by compute scarcity?

Question

Been discussing this in my network, and it seems everyone has a different interpretation of what we mean by "compute scarcity". Some have pointed out that GPUs are in fact heavily under-utilized and just sitting idle.My understanding previously was that it was power/electricity issue (at least in the states). Hence why ppl in the countryside are disillusioned with the big tech since power is diverted to data centers to feed GPUs and cooling systems and they are either having power outages or their bills are going up (or both). Is that true though?You also need real estate - you can't just build a data center anywhere. Ignoring the cooling issue for a second, it could be the reason why Google, SpaceX and others are considering orbiting data centers.What's your take? Does anyone have more info? Is it actual GPU/silicon scarcity or power limitations or something else?

ethigent · Accepted Answer

I would say it is the "metrics driven environment" surrounding Datacenter operations. Which is already sliced and diced to conform to Rack U-age and Cage space. So customers within said Datacener can be charged or each element. Look for water to become a true metric as well!That said all of this primarily rolls up into token usage and AI. Since use of AI resources is based on token usage, each element can also be sliced and diced down to the token cost. A very fair metric frankly, but contributing to compute scarcity as an overall concept.