logoalt Hacker News

agentultra • 01/21/2025 • 3 replies • view on HN

So humans are supposed to review all of the code that GenAI creates. We’re supposed to ensure that it doesn’t generate (obvious?) errors and that it’s building the “right thing” in a manner prescribed by our requirements.

The anecdotes from practitioners using GenAI in this way suggest it’s a good tool for experienced developers because they know what to look out for.

Now we admit folks who don’t know what they’re doing and are in the process of learning. They don’t know what to look out for. How does this tech help them? Do they know to ask what a use-after-free is or how cache memory works? Do they know the names of the algorithms and data structures? Do they know when the GenAI is bullshitting them?

Studies such as this are hard but important. Interesting one here even though the sample is small. I wonder if anyone can repeat it.

Replies

probably_wrong • 01/21/2025

> Do they know when the GenAI is bullshitting them?

Anecdote from a friend who teaches CS: this year a large number of students started adding unnecessary `break` instructions to their C code, like so:

  while (condition) {
    do_stuff();
    if (!condition) {
      break;
    }
  }
They asked around and realized that the common thread was ChatGPT - everyone who asked how loops work got a variation of "use break() to exit the loop", so they did.

Given that this is not how you do it in CS (not only it's unnecessary, but it also makes your formal proofs more complex) they had to make a general one-time exception and add disclaimers in exams reminding them to do it "the way you were taught in class".

➕ show 4 replies
diggan • 01/21/2025

> Now we admit folks who don’t know what they’re doing and are in the process of learning. They don’t know what to look out for. How does this tech help them? Do they know to ask what a use-after-free is or how cache memory works? Do they know the names of the algorithms and data structures? Do they know when the GenAI is bullshitting them?

You can know enough in X to allow you to do Y together with X, which you might not have been able to before.

For example, I'm a programmer, but horrible at math. I want to develop games, and I technically could, but all the math stuff makes it a lot harder sometimes to make progress. I've still managed to make and release games, but math always gets in the way. I know exactly how I want it to behave and work, but I cannot always figure out how to get there. LLMs help me a lot with this, where I can isolate those parts into small black boxes that I know they give me the right thing, but not 100% sure about how. I know when the LLM gives me the incorrect code, because I know what I'm looking for and why, only missing the "how" part.

Basically like having 3rd party libraries you don't fully understand the internals of, but can still use granted you understand the public API, except you keep in your code base and pepper it with unit tests.

squigz • 01/21/2025

> Do they know to ask what a use-after-free is or how cache memory works? Do they know the names of the algorithms and data structures? Do they know when the GenAI is bullshitting them?

No, which is why people who don't pick up on the nuances of programming - no matter how often they use LLMs - will never be capable programmers.