I think that also one needs to be familiar with modifying the guix daemon service to allow substitutes be downloaded so that you don't have to build derivations [0], [1].
Once you modify the guix daemon service and approve the substitute server keys with `guix archive`, your experience should be much faster.
[0]: https://guix.gnu.org/manual/en/html_node/Substitute-Server-A...
[1]: https://guix.gnu.org/manual/en/html_node/Getting-Substitutes...