Well, the idea here is “things that look like buttons do page navigation.”
And the proposal is that we should more closely align the presentation and function.
You are correct that if you don’t care about alignment, there is nothing wrong with the status quo.
Presentation in what medium? A screen reader? You’re talking the user-agent stylesheet, which is a particular chosen presentation that I can change with my own CSS. I can make a button look like a link.
You’re suggesting aligning to an inherently and deliberately changeable presentation.
The idea is fine, but worry about what an HTML element “looks like” is not a good reason.