The close button has always been there, you double clicked the top left menu button. That worked all the way until Microsoft started redoing window decorations in desktop mode with Windows 8.1, and even for a short period after.
This was also copied into other X window control styles. Even today, a Motif replicates the Windows 1.0-3.11 top-left menu+close button.
And that answer is precisely why (1) Windows 95 was such a revelation to the market and (2) nerds like us remain oblivious to that[1] even three decades on.
Yeah, yeah, I know CUA allows for a window close. No one knew. I worked IT at the time (as did lots of us here in our youth I'm sure) and was constantly teaching and re-teaching this trick to the poor people trapped with their CUA environments.
But suddenly with Windows 95 you could see how it worked.
[1] Even if we knew in our bones, c.f. this very discussion about the popularity of a cloned hack on Linux, that it was the Right Thing.
It actually still works to this day. Doubleclick the top left menu button (the one with the app icon) and your window closes.