This model I think will be too slow for that on Spark, even at 4 bit quant.

It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited

I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size:

NVFP4 Q/K/V/O and MLP projections: ~13 GB/token

BF16 attention gates: ~3 GB/token

BF16 LM head: ~2.5 GB/token

Total: ~18.9 GB/token

At 273 GB/s, that gives a bandwidth-only ceiling of about 14.5 tok/s; actual performance would be lower.