logoalt Hacker News

xkcd-sucksyesterday at 10:25 PM6 repliesview on HN

Domain expertise combined with a QA mindset could replace SWE, but consistent QA mindset is rare


Replies

notRobotyesterday at 11:02 PM

I agree that a consistent QA mindset is rare, but I'm not sure even if present if it's enough to replace an SDE.

I very recently looked at the codebase of a vibe-coded app made by someone with domain expertise but no software dev experience.

It was very clear to me that he had described it from his POV to an AI, and the AI had implemented features in a manner that technically worked, but made future maintenance or expansion extremely tricky, which is why he was now looking for a dev.

For example, in his data schema, for every item on a menu, instead of simply having an array property like so for ingredients:

    items["latte"]["ingredients"] = ["water", "milk", "sugar", ...]
He had individual flags for every item for every possible ingredient it could have or not have:

    items["latte"]["has_milk"] = true
    items["latte"]["has_nutmeg"] = false
    items["latte"]["has_cinnamon"] = false
    items["latte"]["has_sugar"] = true
    ...
This technically worked and passed tests from his POV at an MVP level. But added a lot of complications when actually trying to build more features or when a new menu item had ingredients the founder hadn't thought to include in the schema beforehand.

I totally get how he ended up where he did though. While describing it to the AI, he probably said something like "store info on each menu item's ingredients, they might have milk or coffee or sugar", and the AI created individual flags for them and he didn't think to question it, because he didn't know what's "right" or "wrong", but then as he kept building the AI stuck with keeping individual flags instead of swapping it out with an array mechanism, and he couldn't have known the correct way to implement it.

Only a dev with experience would know how to describe the system to an AI model to get an output that works well, and how to assess the quality of its output beyond what can be assessed through the basic UI. This wasn't a QA failure, it was a design failure.

show 1 reply
michaelchisariyesterday at 11:03 PM

I disagree. At some point of complexity, building it yourself is faster, better and (as we're finding out) cheaper. And more fun, although that varies person to person.

Wrestling with a code generator also creates a sunk cost fallacy where progress grinds to a halt but you still try and use the tools to fix the problems the tools created. Or you go in and fix things yourself, in a codebase you don't truly understand. A single developer can recreate the contextual nightmare miasma of a large corporation all by themselves.

There's also an emerging market consideration: MVP are easy to build so time to market is no longer hard to achieve. It's not a differentiator.

X was built in 3 days but is slow and riddled with bugs and security errors. There are also A, B, C, D and E which are effectively the same thing built just as fast.

Z was built over six months and is rock solid and performant.

Who wins the market share?

show 1 reply
eggplantemoji69yesterday at 10:47 PM

Personally my ability to understand atrophies / is reduced when compared to writing code ‘myself’ rather than fully being a reviewer.

Probably similar to hand writing notes (while digesting + synthesizing and not just being a scribe) vs reading notes somebody else took.

show 1 reply
nradovyesterday at 11:28 PM

You can't test quality into a product. Regardless of how much of a "QA mindset" you have, you can only ever find a fraction of defects and technical debt through external testing. This can be good enough for a throwaway app that will only be used by a limited customer base for a limited time. But that approach quickly bogs down if you try to scale it into a product that will be used indefinitely by a huge set of external customers. At some point velocity drops to near zero because the code base is such a mess that it becomes impossible to add new features without causing regression defects or breaking backward compatibility.

goosejuiceyesterday at 11:28 PM

The engineering part of software engineering is the hard part for LLMs. How is that replaceable with these skills?

rustystumpyesterday at 10:43 PM

I don’t think so. Most things are sufficiently complicated enough to require multiple domain experts working together to achieve a goal.

The dunning kruger effect is in full swing as people think AI replaces the domain expert need.

Most of the value in the expert isnt the 80% but the tail 20% or 10% where AI fails. For a one of personal app or website, 80% is plenty but only that.