I think the difference here is between your conception that reality follows a mathematical model while their conception is that mathematical models follow and try to be reflective of reality.
I think their concern is that, if one believes reality follows math, when the model fails to accurately predict something, the person with the model may wonder what’s wrong with reality. If that person believed the model follows reality they would wonder what’s wrong with the model. The latter perspective will yield better results.
It’s the difference between saying “this is how it works” vs “to the best of my knowledge this is how it works.”