logoalt Hacker News

dakshgupta11/07/20241 replyview on HN

Love the demo video! Three quick questions:

Any specific reason to choose the terminal as the interface? Do you plan to make it more extensible in the future? (sounds like this could be wrapped with an extension for any IDE, which is exciting)

Also, do you see it being a problem that you can't point it to specific lines of code? In Cursor you can select some lines and CMD+K to instruct an edit. This takes away that fidelity, is it because you suspect models will get good enough to not require that level of handholding?

Do you plan to benchmark this with swe-bench etc.?


Replies

jahooma11/07/2024

We thought about making a VSCode extension/fork like everyone else, but decided that the future is coding agents that do most of the work for you.

The terminal is actually a great interface because it is so simple. It keeps the product focused to not have complex UI options. But also, we rarely thought we needed any options. It's enough to let the user say what they want in chat.

You can't point to specific lines, but Codebuff is really good at finding the right spot.

I actually still use Cursor to edit individual files because I feel it is better when you are manually coding and want to change just one thing there.

We do plan to do the SWE bench. It's mostly the new Sonnet 3.5 under the hood making the edits, so it should do about as well as Anthropic's benchmark for that, which is really high, 49%: https://www.anthropic.com/news/3-5-models-and-computer-use

Fun fact is that the new Sonnet was given two tools to do code edits and run terminal commands to reach this high score. That's pretty much what Codebuff does.

show 1 reply