Shallow understanding from people of goodwill is more frustrating than absolute misunderstanding from people of ill will.
I'm not going to say it any better than MLK, so I'll just leave the link here.
White America is the problem. Sure, moderate white Americans are so much less terrible than outright racists, but a complacent majority that is happy to reap the benefits of an unjust system, simply because it is the comfortable path, is one of the major reasons why it's so hard to make movement against those racists.