> You're correct in principle, but in reality...

... It's never been tried. Look, I was actually joking when I initially replied, but now that I think about it, it's never been tried!

So now I am left wondering exactly what will happen if I train a new model from scratch, on only a single short corpus of 50k words.

Not fine-tuning an existing model, but literally creating a new model by feeding a blank (or random) matrix the same poem 100k times.