logoalt Hacker News

herpderperatortoday at 5:45 AM0 repliesview on HN

I'm curious: If someone wanted to serve models off hardware/silicon directly (like Cerebras or Taalas, and soon Google I think) rather than GPUs, would these parameters still be adjustable at request time? Or would they have to decide that before the model can even start serving and it would be locked in until they reload it (which would make it briefly unavailable)?