alt Hacker NewsI mean, I'm curious what kind of code it's saving you time on. For me, it's worse than useless, because no prompt I could write would really account for the downwind effects in systems that have (1) multiple databases with custom schema, (2) a back-end layer doing user validations while dispatching data, (3) front-end visual effects / art / animation that the LLM can't see or interpret, all working in harmony. Those may be in 4 different languages, but the LLM really just can't get a handle on what's going on well enough. Just ends up hitting its head on a wall or writing mostly garbage.
I have not memorized all the docs to JS, TS, PHP, Python, SCSS, C++, and flavors of SQL. I have an intuition about what question I need to ask, if I can't figure something out on my own, and occasionally an LLM will surface the answer to that faster than I can find it elsewhere... but they are nowhere near being able to write code that you could confidently deploy in a professional environment.
Replies
I've also not had great experiences with giving it tasks that involve understanding how multiple pieces of a medium-large existing code base work together.
If that's most of what you do, I can see how you'd not be that impressed.
I'd say though that even in such an environment, you'll probably still be able to extract tasks that are relatively self contained, to use the LLM as a search engine ("where is the code that does X") or to have it assist with writing tests and docs.
I’m far more in the camp of not AI than pro LLM but I gave Claude the HTML of our jira ticket and told it we had a Jenkins pipeline that we wanted to update specific fields on the ticket of using python. Claude correctly figured out how we were calling python scripts from Jenkins, grabbed a library and one shorted the solution in about 45 seconds. I then asked it to add a post pipeline to do something else which it did, and managed to get it perfectly right.
It was probably 2-3 hours work of screwing around figuring out issue fields, python libraries, etc that was low priority for my team but causing issues on another team who were struggling with some missing information. We never would have actually tasked this out, written a ticket for it, and prioritised it in normal development, but this way it just got done.
I’ve had this experience about 20 times this year for various “little” things that are attention sinks but not hard work - that’s actually quite valuable to us