Since my polymorph meme has only garnered three downvotes so far I thought I'd offer a bit more controversial take, and see if I can manage to stir the pot a bit with this one.
I’d allow it, but instead of HP damage, you take as many levels of exhaustion.