The problem is that populations still continue to grow and the number of vegetarians/vegans is neglible to overall growth.

any excuse you make doesn't change whether more animals were killed this year than last, regardless of how many vegans there are.