alt Hacker NewsYes there are reasons to create better models. Baking in latest revision of assorted API documentation is not it.
You could simply try it. Find an API that is not publicly documented and thus not part of any training set. Take any AI model, give it the documentation and tell it to create some code that uses that API.
This reminds me of a funny experience at work
They linked some internal llm with all the confluence documentation sets
My product owner tried to find information about our product, as a regular user would, so feel the experience from a user's perspective
It was a failure: only random, generic answers were received
Then, he retried, specifically citing our main confluence pages: it was a success
Conclusion: for the users that already know were is the documentation, the tool is helpful. Reading the page would be also helpful.
Conclusion (reworded): if you know where is the treasure, then the magic gold-detector will work;