> The model weights and inference code need to be contained within 25KB of user-space memory

Wouldn’t it be era-appropriate to allow relying on banked memory? You’d still need to hold the inference code, but you could effectively stream(ing page) the weights as you compute on them.