alt Hacker NewsBragging about "using agents to code pre-GPT" is quite the corporate flex... weren't they notoriously bad back then? Might as well say, "We've been doing spreadsheets with quantum computing since 2016"
Replies
I'm skeptical. Pre-ChatGPT the models didn't understand tool calls or file editing so you couldn't really do targeted edits. So I really am not sure what they mean by this.
I mean, it's not impossible - I wrote a "coding agent" with GPT-3. It sucked balls. The idea was you write a Markdown spec file and the tool then compiled it to an equivalent source code one shotting it every time, but it hardly worked at all. The file changed too much every time and instruction following wasn't good enough, so it'd keep changing the interface exported by the file, and there were lots of bugs etc.
I would not say they were bad, mostly just primitive compared to what we have today. I can only presume they are talking about GitHub Copilot which was released before ChatGPT. I believe at the time it was little more than “spicy autocomplete.” Yet it could be surprisingly capable, e.g. I recall this blog, also pre-ChatGPT: https://medium.com/data-science/github-copilot-crushes-data-...
That said I did happen to work with a team in ‘21 - ‘22 that got access to an older coding model from OpenAI, also called Codex, through a corporate partnership. We also tested it out in an autocomplete UX. Our experience was rather hit-or-miss and I was a bit skeptical of AI-based coding at the time. That blog post above and others like it were an eye-opener and made me wonder if we were just “holding it wrong.”
Then ChatGPT was released. It was nowhere as good as the models today but it could write reams and reams of correct code and I realized the world had changed forever.