Have you ever generated access frequency statistics for the experts in these models, something like a histogram?
ktransformers can do dynamic placement of experts and could presumably produce such a histogram, though currently its activation statistics are just a ".pt" file. https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...
FWIW I never got it to work and did not dig into it much.
alt Hacker News
ktransformers can do dynamic placement of experts and could presumably produce such a histogram, though currently its activation statistics are just a ".pt" file. https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...
FWIW I never got it to work and did not dig into it much.