alt Hacker NewsA UK trance artist called Deathboy left directory-traversal open on his website about 23 year ago. Since then I've had a lot of mp3's that he's never released or put on albums, which is sad because a lot of them are pretty great.
Similarly (also from ~2003), the (Australian) ABC's website held a lot of recorded breakfast radio show clips from when Adam & Wil hosted it, getting the awesome comedy band Tripod [0] to write songs in an hour. Many of these were released on their CD's, but nowhere near all of them.
Eventually that ABC server was shutdown due to lack of government funds. There's a very good chance I'm the only one on the planet with these excellent songs & interviews from those shows.