Representative democracy is beholden to the majority. if they majority voted to kill you, would you just accept it and let them?
Also who are the members? what chooses who is and isn't a member of this body that has the right to your life? A made up line in the sand? Why do I have to rely on the opinion of someone who could be half a continent away? The only people I want to be held accountable to are the ones that I have to interact with, because others don't know me, and they shouldn't have any say on whether or not I get to live.
Anarchy is exactly what you say: being held accountable by other members of your society. It just doesn't try and mash millions of people with different viewpoints together and shrug when they inevitably start killing each other. It allows for dynamic formation of groups that think alike and uses the overlaps to build a network of people.