alt Hacker NewsIIRC, (& I don't remember if I reported it), but Azure's audit logs don't reflect reality when you delete a client secret from the UI, either.
If I remember the issue right, we lost a client secret (it just vanished!) and I went to the audit logs to see who dun it. According to the logs, I had done it. And yet, I also knew that I had not done it.
I eventually reconstructed the bug to an old page load. I had the page loaded when there were just secrets "A" & "B". When I then clicked the delete icon for "B", Azure deleted secrets "B" and "C" … which had been added since the page load. Essentially, the UI said "delete this row" but the API was "set the set of secrets to {A}". The audit log then logged the API "correctly" in the sense of, yes, my credentials did execute that API call, I suppose, but utterly incorrectly in the sense of any reasonable real-world view as to what I had done.
Thankfully we got it sorted, but it sort of shook my faith in Azure's logs in particular, and a little bit of audit logs in general. You have to make sure you've actually audited what the human did. Or, conversely, if you're trying to reason with audit logs, … you'd best understand how they were generated.
I don't think I would ever accept audit logs in court, if I were on a jury. Audit logs being hot lies is within reasonable doubt.
Replies
That's why I'm a great fan of positive confirmation steps before such changes with possibly large implications. The whole change needs to be shown to the user with all changes marked and then you confirm once more that that is what you want and then that and only that gets executed. All these 'video game' interfaces with implicit saves and underwater API calls are super dangerous.
That's crazy and a pretty good point.
The human in the loop doesn't really control what gets done, it only expresses intend to the frontend.
There is so much goofiness happening in those web portals (and also the New Portal, and the Legacy Portal) that issues like this don’t surprise me. Every time I click a button in there I worry that the wrong thing will happen to a different object. Sometimes the display reflects the worst possible outcome, like adding a user to a group will show you the new group membership as just containing that 1 new user and nobody else. Quite a few moments of panic.