logoalt Hacker News

niemandhiertoday at 2:42 PM2 repliesview on HN

My understanding is, that we do not know if the chain-of-thought actually matters in the way we assume for the result.

I think there were experiments where seemingly relevant parts of the CoT were ablated and it did not change the result.

For all we know it might be somewhat human parseable neuralese.


Replies

skybriantoday at 2:53 PM

No guarantees, but it might still be a useful signal even if it’s not causative?

jamesharttoday at 3:05 PM

Or the neuralese could be hidden steganographically in the chain of thought.