Yeah, I agree with that. Very few japanese are religious in the sense that many western countries understand the concept; it's unlikely to be a big force as you rightly point out. There are other factors, but nothing supernatural (though the far right venerating the emperor is a whole other can of worms)