Consensus does matter when it's a consensus of experts in a specific field. When I look at evolution, I follow the consensus of evolutionary biologists. When I look at the historicity of Jesus, I follow the consensus of historical scholars who study that era. I'm not an expert myself, so I have to trust someone else. I think that's true for everyone outside of their expertise.
Plus I would probably agree with you that if a "scholar" believes that Jesus did miracles, I wouldn't trust that scholar.
All I'm saying is that most likely, some guy named Joshua was baptised and crucified, and in between probably did some preaching that inspired a religion. Given that this is the consensus view by experts on the subject, the onus is on others to provide evidence that this isn't the case. But acknowledging that this is the case doesn't threaten my belief in materialism.