I think the usage implies it's so easy to parallelize that any competent programmer should be embarrassed if they weren't running it in parallel. Whereas many classes of problems can be extremely complex or impossible to parallelize, and running them sequentially would be perfectly acceptable.

It's commonly used in some corners of computer science

It's in the same spirit as the phrase "an embarrassment of riches". So a bit of an archaic usage.