There is definitely something wrong with your setup. I can run an amd64 container on my Macbook Pro M3 in well under a second:
[~]$ podman pull --arch=amd64 debian:13
Resolved "debian" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/debian:13...
Getting image source signatures
Copying blob sha256:866771c43bf5eb77362eeeb163c0c825e194c2806d0b697028434e3b9c02f59d
Copying config sha256:a3624ddeb711bef28c29e6de1502fc3ef9df132c220d1db5a121b2a1e2a74256
Writing manifest to image destination
a3624ddeb711bef28c29e6de1502fc3ef9df132c220d1db5a121b2a1e2a74256
[~]$ time podman run --rm -ti debian:13 uname -m
WARNING: image platform (linux/amd64) does not match the expected platform (linux/arm64)
x86_64
podman run --rm -ti debian:13 uname -m 0.03s user 0.02s system 9% cpu 0.456 total