alt Hacker News>They [anthropic] appear to have just told it to get some results and then strapped its nose to the grindstone until it found some.
it is fun how well this works.
i cant find the link immediately (will look and edit with it), but somewhere in the "hello there the jacobian conjecture is false thanx" thread, someone brought up a different conjecture breakthrough where the prompts were basically just repeated "no, keep going" until a result was found.
edit: https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
i especially like "you should do a breakthrough". each prompt is less than ~20 words. makes me really question the whole "prompt engineering" stuff.
Replies
> i especially like "you should do a breakthrough". each prompt is less than ~20 words. makes me really question the whole "prompt engineering" stuff.
This is well past prompt engineering and into process engineering like six sigma. Just like in an early industrial revolution factory, we’re all still figuring out what works in the process of making stuff except this is so early that even simple things like “make this screw standardized” (or “no, keep going” in this case) is really high impact.
The degrees of freedom an LLM has is so large that we're going to be exploring their capabilities for decades, especially if they continue to get better. This is why IMO experts are always going to be better at LLMs in their field because they can force them LLM into processes (think prompt engineering -> CC dynamic workflows) that follow their work processes and get much better results out of them than “keep going.”
In response to your edit, you should check out Terry tao's chat gpt logs about the recent Jacobian result. The models are smart enough to brute force some things, but can cut to the meat much faster with good prompting
That second person stated that for many years they tried that particular graph problem on various AI models, starting with o1 and o3.
Its quite likely they now found the counterexample with a more serious prompt, and then for virality re-tried a few times with meme-prompts like "you should do a breakthrough", knowing that the model is capable of solving this particular one. Worst case the meme-prompts don't work and they share the real one they initially used.
Oh boy. The human here is effectively assuming the role of a Magic 8 Ball…
What a weird species of halting problem…