alt Hacker NewsA lot of distillation happens. E.g. OLMo models have a completely open dataset and they are heavily distilled. It only makes sense to try to absorb behaviors from the best models out there. That said, I think the open weight juggernaughts are doing really genuinely great work with RL, training environments, architectural innovations etc.
Thanks for the response. i had too many noodles tonight and forgot to check my writing. I’m a rare generalist and so it is so very hard to keep up with this without saying “better autocomplete” my one goal is to not get washed out like my parents did in the great username and password wars. i used to have this theory about knowledge in society/silos and i likened it to condensation on a window. you have all this water so close to each other and yet not touching-then, something happens and a bead runs down the window and it all connects. i guess distillation reminds me of it but ai overall reminds me of it. because we all know there are silos and complementary info just waiting to run together and make something happen. I am undoubtedly a naive optimist and believe there are good things coming. it’s not a popular opinion and i think that’s mostly because people would rather spend their time guarding than defining their future. oh baby, there are more noodles in the fridge and to think i almost left them at the restaurant.