alt Hacker NewsThe biggest problem is not that bugs are migrated with COBOL, but that lots of new bugs are going to be introduced. AI is not deterministic, it will be making tons of mistakes. The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools. However, it requires much more time as each step must be prompted, tested and committed manually. Any hope that one-shot migraton of a large code base will not introduce enormous number of bugs is very naive. LLM is very bad on handling long context - it is their nature unfortunately. There is no answer to this problem yet.
Replies
> AI is not deterministic, it will be making tons of mistakes.
Just set the sampling temperature to zero and remove any unintended non-determinism during the parallel computation of the token probability distribution. The problem is solved? Of course, not. Non-determinism has little to do with LLMs' mistakes.
If a considerable size of test data is available for any system, rewriting a well-understood one is much easier today than rewriting by hand. The issue is mostly with the “well-understood” part, as over the years, none cared about understanding and it became a working blackbox that is responsible for a critical part of large system and none wants to take the risk.
You don’t need the entire codebase in context in every moment to migrate it. Also AI being non deterministic does not prevent it from one-shotting perfect solutions 100% of the time for simple enough problems. And every model generation brings this bar higher. So that’s really not a fundamental problem. And we can also implement llm inference deterministically if we want, it’s just that it’s not worth the loss in performance to do it.
> The only realistic low-error approach is incremental step-by-step migration using Cursor or similar tools.
Correction: AI is not deterministic, the only realistic low-error solution is not a more complex use of non-deterministic AI, but deterministic transpilation.
The problem is that this results in COBOL-in-Java which runs correctly but it is a nightmare to maintain.
[dead]
Does it? I found llms to be great for straight conversions
At least if tescoverage is good, but well... That's something llms can also be used for
> AI is not deterministic, it will be making tons of mistakes.
From the paper:
> The COBOL source is passed through an internal deterministic Migrator to produce a generated Java target.
Also, humans are not deterministic either. Give the same COBOL -> Java translation to multiple developers and each will come up with a different solution. Heck, even the same developer will produce a different output for the same task, depending on the day of the week.