logoalt Hacker News

wongarsuyesterday at 8:25 PM0 repliesview on HN

Found it: https://news.ycombinator.com/item?id=47500709

Part 3 might be the best introduction: https://dnhkng.github.io/posts/sapir-whorf/

tl;dr: Based on experiments with similar prompts translated to different languages LLM layers group into three phases: the first decodes from the source language into an abstract space, the middle does something, then there's a last part where the abstract result gets transformed back to the target language. And you can repeat the middle to get a stronger model. Which neatly fits Anthropic's findings here that something similar to CoT is happening in those middle layers

Three months ago. I wonder if Anthropic's J-Space research was actually inspired by those blog posts