It depends on who you ask. The most committed vegans say yes, the less committed shrug their shoulders and don't worry too much about it.
Can you guess which kind I am?
Well you're wrong. I'm not vegan. I had to quit due to health reasons.
Anyway, it's not like being vegan is a religion; avoid animal products as much as you want. Every little bit helps, but I wouldn't beat yourself up over the small stuff.
Personally, I walked around in leather boots the whole time I was eating vegan, because leather is an extremely durable material with no suitable non-animal replacement. I still have those boots. I would have gone through multiple pairs of shoes were it not for those boots. I am very thankful to the cow or bull that had to give up its life so I could have those boots.
Judging by the rest of the comments, yeah, not gonna be a popular answer.