alt Hacker NewsCan a model be stripped off anything not relevant to coding and get a lot lighter? Or is that impossible?
Just like we have professors with specialisation wondering if AI models can also be so.
Replies
I guess it can but it will be useless. After all the model superpower is awareness and ability to guess and infer some stuff. Right now a model saves you time not only by coding faster, but that it can figure out some stuff about the shape of the data and its purpose.
If you throw general purpose model at a codebase - it will look at the table and data logical connections beyond what is explicitly declared. It will figure out on its own that Salaries should be displayed on SalariesTable.php and it will "know" that your prices should include vat and so on.
A human knows that VAT and price go together and are related, full size LLM does too, stripped one - doesn't.
You can... but the trick is to do so without killing performance. Turns out a lot of random things help make coding performance good.