logoalt Hacker News

pixelesquelast Wednesday at 6:07 PM1 replyview on HN

> Using 16-bit floats for lat/lon gives you accuracy down to 1 meter.

Not for Longitude it doesn't with values > abs(128), as that for example means 132.0 has the next possible value of 132.125.

float16 precision at values > 16 is pretty poor.

Converting that discrepency (132.125 - 132.0) to KM gives 10 KM.

Did you maybe mean Fixed-point? (but even then that's not enough precision for 1m)


Replies

tantalorlast Wednesday at 6:20 PM

Good catch, I didn't consider that.