alt Hacker News> Lately, I just steal embeddings from big models and slap a dumb classifier on top. Works better, runs faster, less drama.
You may know this but many don't -- this is broadly known as "transfer learning".
> Lately, I just steal embeddings from big models and slap a dumb classifier on top. Works better, runs faster, less drama.
You may know this but many don't -- this is broadly known as "transfer learning".
Is it, even when applied to trivial classifiers (possibly "classical" ones)?
I feel that we're wrong to be focusing so much on the conversational/inference aspect of LLMs. The way I see it, the true "magic" hides in the model itself. It's effectively a computational representation of understanding. I feel there's a lot of unrealized value hidden in the structure of the latent space itself. We need to spend more time studying it, make more diverse and hands-on tools to explore it, and mine it for all kinds of insights.