Model Training as Code

Some good stuff here from Dwarkesh around mashing up training and inference:

https://youtu.be/20p5-kQXF_Q?is=72ImTNxkOEKmOXQ9

He predicts this kind of model factory will become central to organizational learning and operations. Updating and upgrading the model stack becomes the core staff function.

What is this "aleph" thing in names now? First aleph neuro, and now aleph alpha.

> TL;DR: Model training has grown complex

So they’ve built Savanah - a workflow engine because the existing zoo of hundreds of workflow engines didn’t cut it :)