It says it needs to refresh external IP but I only use it locally so I cant see why it would need this and I have location access allowed so it should know I am home.

The HA app on your smartphone needs to refresh its exernal ip address (not your pi). It still thinks it's on mobile connection (or even is if your wifi coverage is odd) or at least in a different network than your pi running HA (vpn?).

For me, closing and re-opening the app works if I encounter that message.