logoalt Hacker News

hsfzxjytoday at 10:31 AM0 repliesview on HN

Thanks! I can observe similar behaviors on my laptop. FP32 operations are sometimes siliently lowered to a sequence of (FP32->FP16)->(FP16 OP)->(FP16->FP32), causing precision loss. It further frustrated me during npunlock development. Changing custom SHAVE OP from FP16 -> FP32 also changes the blob structure, and I have to explore different binary patching strategy.

As for the interesting cost variance for reshape you've mentioned, I guess surrounding context is the cause. The NPU compiler might adjust the data layout to match successive OPs' requirements. But, yes, that's annoying~

Studying DPU is not my current priority, but I may dig it up in the future to understand its invocation descriptor, and hopefully to discover more interesting stuff. I hope this project ends up being useful to you!