Hard disagree on that. That’s assuming the intentions of the people.

No it's not. It's acknowledging the fact that campaigning politicians largely view people as one of the following:

• With us
• Against us
• Someone we might be able to persuade to be with us
• Not relevant

If you don't vote, they categorise you in that last one. It doesn't matter what's in your heart, what matters is how they perceive you. The only tool you have to change that is your vote.