As an alternative, perennial HN recommendation atuin (https://atuin.sh) logs time, duration and exit code (among other data) for every command.
That way you only have to look at it when you need it, and you can also figure out what you were doing last week/month/year if necessary.
After mucking around for an hour trying to get this information into my prompt, I realised atuin already had it.