Also you can just give advantage if it's such an obviously threatening situation.

That should easily make up for the lost + in Charisma modifier.