I didn't have a local GPU, so I asked it to go out and find hardware. It found a google TPU v6e which seemed reasonably priced. I gave it my google api key. I told it to use TPU only when training and bring it down afterwards. That's about it.
What kind of observability did you have over this process? I’m interested in how my peers are operating these efforts.
> I told it to use TPU only when training and bring it down afterwards.
I wouldn't put my house on it. Brave.
Neat!
> I gave it my google api key
This is the part where the narrator looks at the camera and says "Don't try this at home, kids!"