logoalt Hacker News

AmazingTurtle • today at 11:04 AM • 2 replies • view on HN

so i did some apples to bananas comparison.

i let 4 different agents review the same dirty code base (200+ files changed, c++, game engine):

- deepseek v4.1 flash via deepseek API: took 20m to find one p1 issue that was relevant and a p2 issue that was irrelevant

- glm 5.3 flash NVFP4 via TP2 DGX Sparks: took 1h 5m to find a p3 documentation issue

- gpt 5.6 sol: took 5m30s for one p1 and one p2 issue. both relevant

- gpt 6.1 sol: took 4m30s for two p1 and one p2 issue. both relevant.

Worth to mention: none of those findings overlapped.

I let deepseek v4.1 flash and gpt 6.1 sol both plan fixes for the summary of the findings. in this particular scenario: gpt 6.1 sol planned to increase a buffer size from 128 to 256 to fit the 191 elements that blew past a test scenario. it included some headroom. deepseek v4.1 flash on the other hand decided to make it exactly 191 to satisfy the test, for example.

The TPS decode on deepseek is impressive. I never had a second / feeling of being left aone and though "damn this thing is working". Despite all my efforts, it took ~4x longer in a review, found less and planned worse than GPT 6.1 Sol. Wall clock time. I don't have token counts for comparison. If I were to grade those models from 1-10 (10 = top tier GG), I'd give deepseek a solid 6.5 and 6.1 sol a 7.5-8.0 maybe.


Replies

jdthedisciple • today at 12:43 PM

Was that the old DS 4.1 Flash or 4.1 Flash 1017?

➕ show 1 reply
ProphetOfParado • today at 12:03 PM

What is p1, p2?

This read like a conversation you were having in your head -- unfiltered.

➕ show 1 reply