Ok, at 50k context its about 126 prefill, 13 generation.
Oh, interesting. So beats on prefill and matches at decode. MTP will help you a bit once you have it. I've got a lot of work to do to optimize prefill.
Kinda wish I had a Strix Halo here to play with as well.
alt Hacker News
Oh, interesting. So beats on prefill and matches at decode. MTP will help you a bit once you have it. I've got a lot of work to do to optimize prefill.
Kinda wish I had a Strix Halo here to play with as well.