Right, I don't agree that it should be a legal consequence.
I do however think it should be a moral consequence, a judgment visited on those who would cast such a ridiculous vote. The best way I've heard it termed, is that Trump isn't merely unqualified, he's unqualified squared, which is to say, if you think he's fit to be president, then you also aren't qualified.