alt Hacker NewsI strongly disagree ;-)
The paper's line of reasoning seems to continue the endless subjective loop of assuming your viz framework has the right abstractions & defaults , which the next person will rightfully disagree with for their slightly different eval set
We found in practice:
- LLM's generate charts fine
- LLM's tweak charts fine
- LLM's take user feedback to tweak them fine
In that sense, going higher-level for abstractions, as is being argued for here, is strictly worse: it's better to give controls so the LLM can go deep and customize
In practice, we found the choice of json config language X vs json config language Y to be pretty equivalent across different charting systems (vega, plotly, perspective, etc), LLM's do them all fine
The harder parts were deciding what a good chart is (model, reasoning, context), and opposite of this approach, giving lower-level facility for doing user change requests on tweaks, interactivity, and tricky in practice, when they have a lot of data on it.
You are absolutely right. But note that we are actually on the same point here.
This is exactly why this is an intermediate language designed to get 95% stuff right easily (for expressiveness and reliability purpose), while 5% of more advanced case where the agents need to revise chart for other purpose can be done easily on top of the compiled low-level spec (low in terms of Vega-Lite etc, not SVG). We are not really designing a higher abstraction to replace existing ones.
In the past, the split is like 50% good at first run for some common stuff, all other stuff requires agent-loop or user involvement.
Our goal is to make it easy for most case, not everything needs a full multi-round trip agentic workflow to solve. :)
We are kinda all advanced users in fact, for a lot of users, they are easily get confused with the first time result if that is not as good, and the interactivity cost / multi-round isn't an option.