Correct but you’d also still have that drag just from the kernel dirtying those caches in the first place.

But I was clarifying because the wording could be taken as data/instruction cache and there generally isn’t a full flush of that just to enter/leave kernel.