Apple M series is also aarch64 architecture, isn't it? Could you explain more why you expect Ampere to be slow but M series to be fast?
Ampere Altra is for cloud/datacenters/servers where multithreaded throughput is approximately all that matters. Apple M series is for consumers.
Because they're designed for different things.
Ampere's primary focus is running lots of simple tasks concurrently, at relatively low power, with lots of I/O. So, many tens to hundreds of cores, not too fast, at lower power draw than amd64, with lots of PCIe lanes for storage and network.
Apple's primary focus is user experience and power efficiency. That's why you'll find a handful of fast performance cores and low power efficiency cores, along with graphics acceleration to drive high resolution displays.
Apple design their own Arm-compatible cores from scratch. Ampere use a modified Arm Neoverse N1 core. In addition, the Ampere server that Marcin is using is about 6 years old, and would have been tuned for core count over single thread performance (good for web serving). Basically Arm's own cores aren't nearly as good as Apple's at the best of times, and having a 6 year old server makes things even worse.