More evidence that the original Transformer authors didn't really know what they were doing, but they did have access to more cheap compute than anyone else.