The driver that initialises your plug-in GPU is shipped in flash on the card, is signed by Microsoft, and won't run unless that signature validates.

The GPU is initialized earlier, so that the screen turns on. The GPU driver can access main memory through the bus.

If you let arbitrary code run before you start checking, you don't have a secure boot chain.