All this kWh and Cal nonsense could have been avoided if we just used the SI unit “joule” and its multiples for all energy

Yes! J/s gives so much more intuition than W. Even kJ/h is fine. Using W and kWh is like using knots and measuring distance in knot-days.
I might even prefer Coulombs per second over Ampère, so you immediately see that it's a speed-like property.