LLMs are solving outstanding maths problems by trying a load of stuff until something works, which is exactly what they do in software. If you want a system built out of square blocks jammed into round holes they'll make you one in no time, but if you're working in an area where non-functional requirements actually matter (there are lots) the slopnami is a living nightmare.