> Once you use up the entire internets worth of stack overflow responses and public github repositories you run into the fact that these things aren't good at doing things outside their training dataset.

I think the models have reached that human training data limitation a few generations ago, yet they stil clearly improve by various other techniques.