> and the remainder of VRAM is occupied by the KV cache and whatever experts still fit.

Those days engines try to optimize "whatever experts" part and can use remaining VRAM as cache, moving hot experts to GPU during inference (but less used experts keep executing on CPU, yea).

e.g. llama.cpp recently added flag for that: https://github.com/ggml-org/llama.cpp/pull/29887