alt Hacker NewsI'm OK going through the learning itself (will definitely check out the book though, since it's so well liked!). The question is more, what do you do with that once the book or course is done?
For a junior coder, there's usually a path to recommend, joining some open source project or doing some home scripting, etc. Something where you can be useful even if it's not your full time job.
I'm wondering whether there's some similar hobbyist path available here, or whether there's too much of a jump between the educational/pop sci stuff and the professional realm. If it's the second, that's also fine, I just want to know if I'm missing something.
Replies
The comment below/above is right. Pre-training isn't the path (but you really do need to learn how it's done). Learn how it works, learn the architecture, then take models and do post training. Fine tuning and RL on them. There was a fun/clever/useless project recently posted here about building a terminal emulating model thing, as an example.
One practical-ish thing you could reasonably do with this (beyond just better understanding how models work so you can make better choices/tradeoffs), is post-train some of the small-to-medium models, fine-tune them on your own data or priorities.
You don't need to change many weights, relatively, to add capabilities or modify behavior. Look up LoRas for instance.
I'm of the opinion that post-training existing models is an area with a lot of benefit and fun in the near future (it's already being done here and there of course).
I doubt you _really_ have to know everything in this course to know how to do that, but I'd want this as background personally for it.