This is the thesis of our research paper here, a good middle ground is necessary: https://arxiv.org/abs/2501.04580