I'm reminded of a phenomenon in the 70s and 80s the computer is never wrong in which pricing mistakes and bank errors were expected to be impossible since there was a computer involved.

As an aside, I wonder if this is in any way related to the rush of patents in the 90s and aughts, for things humans obviously do, but on a computer or on the web like transferring money or making transactions. We still have lawsuits like that.

Also related, the predictive policing software that some US counties bought, unvetted, and is used to justify longer sentences for poor and nonwhite convicts so that no judge has to attach his name to bigoted rulings.

We humans seem to imagine that since there's a magic box involved in the computation of our answers that the answer is automatically more precise. Perhaps it's related to the notion that were considering more factors, but that only works if we've properly measured those factors and applied them appropriately to the model. Otherwise, as the saying goes (also from early computing) Garbage in; garbage out.