I'll be your server today: A novel wafer design for a better data center solution

As data center construction costs and challenges rise exponentially, maybe it's easier to redesign the server rather than the buildings they're stored in.

This could either be a billion-dollar idea or a complete failure. However, as data centers begin to reach hyperscale, the construction challenges - and subsequently the costs associated in overcoming them - are soon becoming prohibitive.

No matter how fast we innovate, the bottleneck is implementation. We simply cannot produce enough of everything fast enough; and even if we can, we're still relying on archaic infrastructure which requires significant modernization first. Without that investment, we don’t stand a chance at keeping up with the future demand that will be required. Therefore, my take on this is that perhaps we've got to look at a different part of the installation in order to make this a more viable solution for the time being.

So hear me out: how about instead of redesigning the whole data center to suit all the servers - the construction, cooling, power distribution, networking, everything…

Why don’t we just redesign the servers so that the roadblocks being faced don’t even exist in the first place?

---

I completely accept that any amendment to the design cannot impinge on it’s ability to perform as they currently do; so we’ll start with a brief, and then my solution may make a little more sense:

• Data centers have two main roadblocks in the construction - firstly, they get sodding hot; and secondly, they require a ludicrous amount of power. For a single AI hyperscale facility - it can use 20+ times the amount of power that New York City can use (the whole of NYC uses about 5GW; an AI hyperscale data center can use over 100GW).

  • So our server design or chipset architecture needs to be able to function ideally on less power.

  • Or, at the very least - the design needs to be able to run cooler. If the building has a lesser requirement for an extensive HVAC system, that’ll reduce a good lump of the power to run the building.

• We need to be able to stack them high and wide. Historically, two main types of servers are used - blade servers and rack servers. Both are favoured for a largely modular approach as an efficient way of cramming as much computing power in as compact space as possible, and also being easy to scale. If more capacity is needed, another server can just be installed into a rack.

  • Our solution also needs to be a modular design that’s an efficient use of space. (spoiler alert: if you were expecting anything circular, you’re about to be disappointed).

One of Google’s data centers in Nebraska. This is a traditional configuration for rack servers - deployed in enclosures stacked horizontally [Credit: CRN].

Quite simple really from initial impressions. The most common types of servers currently deployed are rack servers. I completely understand the modularity of them, but some parts of the rack system just don’t make sense to me from a practical perspective. I’m totally willing to be proven wrong and corrected on the reasoning for this, but I’ll cover my main points of confusion below:

1. Second-grade physics teaches every single child that heat rises. We’ve identified heat generation is a huge problem. So why on earth do we stack servers in a rack on top of each other? The further up the stack we go, the hotter those servers will become as they’re subjected to the heat rising from the servers below it. I'm fully aware that solutions have been provided to provide cooling in between the servers in the rack, but this is still a relatively inefficient solution in my view.

   A lot of effort has been put into overcoming the physics, whereas I would suggest it makes much more sense to redesign the server to align with physics. This is to suggest particular culprits for causing heat (the processor and potentially power supply) are put towards the top of each enclosure, and the cooler components are put towards the bottom.

2. More basic physics teaches us that a larger hot object dissipates more heat into its surrounding environment than a smaller one. Therefore, I'm intrigued to know why all components of the server are stored in the same space - the hot power supply and hot processor are stored together, a few centimeters apart; and then they are stored next to other hot power supplies and hot processors.

   This is to suggest an intrigue as to why the majority of the transformers for power supplies cannot be stored away from each server rack in a separate plant room, simply served by a cable, akin to a laptop charger. This will effectively split the heat production between two rooms and make the thermal management requirements much less in each respective space. I accept this isn't the biggest challenge to overcome; and it could result in equivalent requirements - but is perhaps a consideration that may yield some success.

3. The active cooling obsession. It appears that every solution to cooling that is required seems to be some form of active management. A fan that sounds like a Boeing 737 trying to take off or an expensive liquid cooling solution.

   The most recent iterations of direct-to-chip cooling utilizing glycol and other compounds are perhaps the most efficient iteration presently, but I would suggest they are far better than methods of passively dissipating heat through a refined choice of material and geometry to better suit this purpose.

4. Speaking of materials, why is everything made out of thin sheet steel? If these installations are known to get hot, steel is a relatively mediocre method of dissipating the heat. I accept its cheap, but fundamentally, if the same amount of cost has to be spent in keeping the installation cool, it would be a better investment to design the server correctly in the first place to mitigate ever needing such cooling solution. Therefore, a novel approach will suggest some alternative materials that wouldn't necessarily add large amounts of cost per unit but would produce a unit that’s more efficient in passively cooling itself.

So my solution, which I’m dubbing the ‘wafer server’. I’ll address each point roughly in the same order I mention them above:

• We need that modular goodness - so it’ll remain a flat square that can be packed tightly into a space, in close proximity to other servers.

• But, to help with the heat problem:

  • Spread the heat over a much larger surface area. So instead of a rack server measuring about 800mm deep laid on top of each other; we’ll make ours 2 metres. As the same amount of componentry can be spread over that whole surface area, It can obviously be a lot thinner now (hence the ‘wafer’ name). There may be space for more componentry, to house factorial capacity per server.

  • We won’t store them on top of each other, but upright, akin to blade servers. Very few server rooms have a problem with eaves height, and so there’s no issues there. We’ll stack them side-by-side, stood in lines like soldiers. If we still need to put cooling in between, we have that opportunity if needed.

  • We’ll do our level best to passively cool them with heatsinks, which can be finned for efficiency - similar to air-cooled motorcycle engines. Whilst we’re at it, those could be made of anything that’s a better heat dissipator than thin sheet steel. As these are barely going to be moved, they can be made even from fragile or brittle materials. My initial thoughts is intrigue to see how ceramics could help in this situation.

  • Finally, we reconfigure the servers so that as much as practicable, the processors and other heat-producing chipsets are placed near the top of the wafer. Power supplies are stored in an adjacent plant room with it’s own independent cooling solution - the server racks are simply fed by cables ran from floor level (could be ceiling mounted, but likely issues with condensation running down cables into the servers)

  • How that will look roughly:

    

    

As I stated above - I’m fully prepared for the pushback on this, and I’ve not come at this from any computer science standpoint; but I do find it fascinating that the buildings to house servers are being constructed with bleeding edge technology; and the components in the servers are being upgraded to bleeding edge technology - but we’re still relying on enclosure configurations designed by Compaq in 1993.

Even if it’s not the above, I’m certain there’s a better solution which will help plug the gap whilst infrastructure investment catches up.

TH

You may find these of interest: