You only have the right to modify if you can access the source.

If you got (a snapshot of) the source along with the binary, that's fine, there's no need to keep hosting the source anywhere.

But if the company said "for source, see: our github", then that github has to stay up/public, for all the people who downloaded the binary a long time ago and are only getting around to exercising their right to modify today.

They don't need to post new versions of their software to it, of course. But they need to continue to make the source available somehow to people who were granted a right that can only be exercised if the source is made available to them.

(IIRC, some very early versions of this required you to send a physical letter to the company to get a copy of the source back on CD. That would be fine too. But they'd also have to advertise this somewhere, e.g. by stubbing the github repo and replacing it with a note that you can do that.)