alt Hacker NewsCurrently. I'm sure that you could make a system where the cache values are a superset C of e.g. models A and B where C is probably bigger than max(A,B) but smaller than A+B
Replies
wonnage • yesterday at 8:13 AM
Reddit post so take with a grain of salt but this does seem possible. But unclear whether this is actually a viable architecture https://www.reddit.com/r/LocalLLaMA/comments/1t8s83r/nvidia_...
It's equal to A+B. There is literally no sharing possible.