The problem with LLMs in finance is the same as it is in writing, design, and many other disciplines: it isn’t code.
Code objectively does what it‘s intended to do or it doesn’t (and passes certain tests or not) which gives coding agents an indication on whether their solution is adequate.
This is much harder in almost any other discipline.
Pass-fail tests in other disciplines are much less useful. You can tell an AI to not use certain words or not write sentences longer than X, but those rules are insufficient.
At no point can a piece of writing or a design be evaluated to “work” the way code does.