Sheffield start-up Iceotope can dramatically cut,by over 90%,the cooling overhead of data centres,and increases the computational density of server racks,by moving to liquid cooling.
Conventional servers are cooled by air,with fans bowing over processor heatsinks.
Resulting hot air is cooled by a heat-exchanger in the back of the cabinet,or air conditioning units elsewhere in the server room.
On top of the power consumed by the server electronics"you have to add at least 20%because of the fans in the rack,and the air conditioning can bring this up to 120%of the electronic load",Iceotope CTO and system inventor Peter Hopton told Electronics Weekly."Some back-cooled air-cooled racks can get down 50-90%of processor power,but they struggle to get 20kW-worth of electronics inside the rack."
Hopton's claims is that he can get 22kW worth of servers into a rack,and get that heat out to the environment,with only 4%extra energy.
Behind the Iceotope cooling technique are two main ideas:keep the heat in liquid,so that air never needs to be handled,and avoid refrigeration,with its power-hungry compressors.
To implement liquid cooling,almost everything in an Iceotope server rack is different,and made in the UK.
The only thing that stays the same are off-the-shelf blade server motherboards,each of which is mounted inside its own sealed hot-swappable'cartridge',which is flooded with primary coolant-a novel liquid from 3M called Novec.
Novec is both compatible with electronics and fireproof,but its most extreme characteristic is its capacity to remove heat through natural convection.
"Novec is very convective,said Hopton."It has high thermal expansivity-20x more than water-and low viscosity,which means it convects 20-40x better than water."
Its thermal capacity is lower than water,said Hopton,but the overall effect is 10-15x better heat removal than if the Novec was replaced with un-pumped water.
To take heat out of the Novec,there is a secondary circuit with water snaking through a channel inside one wall of the cartridge,supplied through two self-sealing fluid valve/connectors at the back.
One litre of water per minute gaining 5?°C is enough to extract 400W of heat from the server motherboard,with only 20W escaping into surrounding air.
"The meander channel is large,so it has a low pressure drop,and the pressure drop of the valve is small too,"said Hopton.
The rack's cooling water comes from a reservoir at the top of the rack(which Iceotope calls a plenum),flowing into a bottom reservoir through 60 parallel paths,one through each cartridge-48 blades and 12 PSUs.
Overall pressure drop through each path is low enough,said Hopton,that gravity is sufficient to push the required litre per minute through the cartridges.
"The only need for a pump is to return 60 litres of water to the top plenum.One 40W pump would suffice.We have two pumps-80W in total,"said Hopton.
And this is the source of Iceotope's headline marketing claim that it can shift 22kW from a cabinet at the cost of only 80W.
Two pumps mean dual redundancy to keep the servers working after one failure.
For the same reason,there are two heat exchangers at the bottom of the cabinet,one per pump,to transfer heat from secondary loop to a tertiary and final water loop which takes heat out of the building.
"Water in from the external circuit can be at up to 45°C,and water out will be at a maximum of 50°C,"said Hopton.
As returning water only has to be cooled to 45°C,this can be done by simple outdoor radiators almost any where in the world,providing that they are in the shade.There is no need for refrigeration and its attendant energy cost from compressors,said Hopton.
As external cooling is passive,the total energy budget is 22kW of heat per rack from the electronics,80W/rack from the rack pumps,plus the energy required to pump the tertiary circuit.
According to Hopton,modelling by a third-party data centre simulation firm predicts that cooling is always possible for a total of 4%over the power budget of the electronics-including pumping twin tertiary loops,and sprayed water evaporative cooling water to boost the outside radiators in extreme weather.
"Even in Houston Texas in mid-summer,we can still get 45°C water back to cabinet with some evaporative cooling,"he said.
Getting the data out of the cartridge
With each server sealed in a cartridge full of liquid,how does the data get in and out?
Power and secondary cooling water are the only rear cartridge connections.
All data exchange is at the front:40Gbit/s Infiniband,Gigabit Ethernet,10 gigabit Ethernet,and PCIe 3.0 x16.
Connections to the server PCB are through high bandwidth polyolefin twinax cable from 3M,which is longitudinally-sealed to stop it acting like a hose pipe.
These run through an epoxy seal in the top vapour space to front-mounted data connectors via buffering and re-timing where appropriate.
Iceotope is not the only way
Other firms are proposing alternative liquid cooling techniques
LiquidCool Solutions from Minnesota is using a cartridge,but pumping the primary fluid,a hydrocarbon,through the back connectors.
Green Revolution Cooling of Texas is immersing many blades in a rack-width vat of another hydrocarbon cooling liquid.
