alt Hacker NewsAh I see! The change helps.
> think that "symbol manipulation" of computers was a kind of category error
This is where we do drift off into the semantic bog. If there is tacit knowledge in an LLM, then where is it? It must be in the weights, and it must have somehow come from the training data. Therefore the weights represent "compressed" knowledge. Is that then not "tacit" since it's explicitly encoded?
You'll get no disagreement from me here. But be ready because some people live to debate this, and Dreyfus is the poster child for a whole philosophical tradition.
I think what it comes down to, is trying to turn the specialness of human intelligence into undefinable magic, essentially playing god of the gaps with the concept of human insight. So by design, it has to be something that can't be amenable to any formal representation like weights.