It's a different diagnosis, but the problem is still, "you transformed my content in a way that changes my appearance and undermines my credibility." The distinction is worth discussing but the people levying the criticism aren't wrong.

Perhaps a useful analogy is "breaking userspace." It's important to correctly diagnose a bug breaking userspace to ship a fix. But it's a bug if its a change that breaks userspace workflows, full stop. Whether it met the letter of some specification and is "correct" in that sense doesn't matter.

If you change someone's appearance in your post processing to the point it looks like they've applied a filter, your post processing is functionally a filter. Whether you intended it that way doesn't change that.