Depending on carrier-specific configuration and firmware phones may be configured to prefer NITZ (time transmitted by the cellular network) instead of NTP. That time is probably what’s off and would explain your observation.