How is this sort of package cooled? Seems like you'd pretty much need to do some sort of water cooling right?
You can move a lot of air with good efficiency even just by using bigger fans that don't need to spin as fast most of the time. Water cooling is a good default for power-dense workloads, but far from an absolute necessity in every case.
Air almost certainly. They always develop these chips within a thermal envelop. The envelop should be within what air cooling can do.
PS. Having many cores doesn’t mean a lot more power. Multi core performance can be made very efficient by having many cores running at lower clock rate.
You can cool it however you want but the better the cooling the better the performance. We'll probably see heat pipes at a minimum.
While the power draw might be high in absolute terms, the surface area is also quite large. For example, the article's estimates add up to just 2000mm2 for the Epyc chip. For reference, a Ryzen 9950X (AMD's hottest desktop CPU) has a surface area of about 262mm2, and a PPT (maximum power draw) of ~230W. This means that the max heat flux at the chip interface will almost certainly be lower on the Epyc chip than on the Ryzen - I don't think we're going to be getting 1000W+ PPT/TDP chips.
From that you can infer that there shouldn't be the need for liquid cooling in terms of getting the heat off the chip.
There still are overall system power dissipation problems, which might lead you to want to use liquid cooling, but not necessarily.
For example, Super Micro will sell you air cooled 1U servers that options up to 400W CPU options (https://www.supermicro.com/en/products/system/hyper/1u/as%20...)