Using watts is fine for anyone who deals with energy and power all the time. The problem comes when the lay person tries to reason about power. If power were written as J/s then they could use the same reasoning that they are already familiar with from dealing with speed and position, or with flow rate and volume.

The real problem is the widespread usage of Wh as a unit of energy

It would make way more sense to use J and J/h instead