alt Hacker Newsand can be faster if you can get an MOE model of that
Replies
bigyabai • yesterday at 9:33 PM
>90% of inference hardware is faster if you run an MOE model.
and can be faster if you can get an MOE model of that
>90% of inference hardware is faster if you run an MOE model.
"Mixture-of-experts", AKA "running several small models and activating only a few at a time". Thanks for introducing me to that concept. Fascinating.
(commentary: things are really moving too fast for the layperson to keep up)