Run strace on Python starting up- you will see it statting hundreds if not thousands of files. That gets much worse the slower your filesystem is.

On my linux system where all the file attributes are cached, it takes about 12ms to completely start, run a pass statement, and exit.