Still rocking a 3090 so can't speak from experience but general vibe around simple at home inference seems like it has improved (esp since both vulkan and rocm are now viable paths on newer cards).

>development using pytorch

Would probably still play it nvidia safe for more adventurous stuff than token generation even if it has improved