I use an extension that gives me a customized homepage, but I still always get the "what's new" tab on every major version upgrade.

It's a totally separate tab that opens. It's got nothing to do with what you use as your homepage.