I use them minimally and haven't stared at enough failures yet to see the patterns. Generally speaking my MO is to remove at least half of the moving parts of any CI/CD system I encounter and I've gone a multiple of that several times.

When CI and CD stop being flat and straightforward, they lose their power to make devs clean up their own messes. And that's one of the most important qualities of CI.

Most of your build should be under version control and I don't mean checked in yaml files to drive a CI tool.

Exactly this. I've used Jenkins, Travis, CircleCI and all of them were so easy in comparison to the github actions runner mess.