logoalt Hacker News

swatcoderyesterday at 8:22 PM0 repliesview on HN

> This kind of work seems like a great use case for AI assisted programming

Always check your assumptions!

You might be thinking of it as a good task because it seems like some kind of translation of words from one language to another, and that's one of the classes of language transformations that LLM's can do a better job at than any prior automated tool.

And when we're talking about an LLM translating the gist of some English prose to French, for a human to critically interpret in an informal setting (i.e not something like diplomacy or law or poetry), it can work pretty well. LLM's introduce errors when doing this kind of thing, but the broader context of how the target prose is being used is very forgiving to those kinds of errors. The human reader can generally discount what doesn't make sense, redundancy across statements of the prose can reduce ambiguity or give insight to intent, the reader may be able to interactively probe for clarifications or validations, the stakes are intentionally low, etc

And for some kinds of code-to-code transforms, code-focused LLM's can make this work okay too. But here, you need a broader context that's either very forgiving (like the prose translation) or that's automatically verifiable, so that the LLM can work its way to the right transform through iteration.

But the transform you're trying to do doesn't easily satisfy either of those contexts. You have very strict structural, layout, and design expectations that you want to replicate in the later work and even small "mistranslations" will be visually or sometimes even functionally intolerable. And without something like a graphic or DOM snapshot to verify the output with, you can't aim for the iterative approach very effectively.

TLDR; what you're trying to do is not inherently a great use case. It's actually a poor one that can maybe be made workable through expert handling of the tool. That's why you've been finding it difficult and unnatural.

If your ultimate goal is to improve your expertise with LLM's so that you can apply them to challenging use cases like this, then it's a good learning opportunity for you and a lot of the advice in other comments is great. The most key factor being to have some kind of test goal that the tool can use for verify its work until it strikes gold.

On the other hand, if your ultimate goal is to just get your rewrite done efficiently and its not an enormous volume of code, you probably just want to do it yourself or find one of our many now-underemployed humans to help you. Without expertise that you don't yet have, and some non-trivial overhead of preparatory labor (for making verification targets), the tool is not well-suited to the work.