alt Hacker NewsTheir preprint showed it worked well with small batch size, but most prod deployments right now have rather large batch sizes from serving multiple users
Replies
gallerdude • today at 7:41 PM
I wonder if it'd work better as a local model
Their preprint showed it worked well with small batch size, but most prod deployments right now have rather large batch sizes from serving multiple users
I wonder if it'd work better as a local model
That's a great answer, thanks. I'd missed that detail.