> 3. I tried the usual diff tools. They diffed line by line and result was terrible.
Um. I don't want to be That Guy (shouting at clouds, or at kids to get off my lawn or whatever) but ... what "usual diff" tools did you use? Because comparing two text files with minor edits is exactly what diff-related tools have excelled at for decades.
There is word-level diff, for example. Was that not good enough? Or delta [0] perhaps?
kompare, kdiff3, meld, git diff --no-index (--patience, --word-diff, --histogram)
All weren't remotely close to what I wanted.
On first glance delta is also doing completely standard thing. But I can't rule out that there are some flags to coerce it to do the thing I wanted. But searching for tools and flag combinations is not that fun and success is not guaranteed. Also I found a (SaaS) tool and a flag there that did exactly what I wanted. Just decided to make my own local tool afterwards for better control. With an agent.
> Because comparing two text files with minor edits is exactly what diff-related tools have excelled at for decades.
True. And I ended up using this excellent Google diff-match-patch library for diffing. AI didn't write me a diffing algorithm. Just everything around it to make a convenient tool for me. Most tools are for source code and treat end lines as a very strong structural feature of compared texts. I needed something that doesn't care about end lines all that much and can show me which characters changed between one line of one file and five lines it was split into in the second file.