It's such a niche tech space. To play a bit of devil's advocate, a properly designed IVR will have "DTMF clamping" which veils the dial tones (the same ones you hear your phone play when dialing a number, did you ever notice the tones are unique?). The IVR should also disable logging completely. When on a call, they should be disabling call recording.
This is part of a process called PCI compliance, and it's fucking huge, because the penalties for it are insane, tens of thousands of dollars per month, plus extra for each incident of non-compliance. Some companies do transactions in the millions, at a $50 fine a pop. British Airways was fined $229 million back in 2017 for exposing data.
So really, companies are always going to do their due diligence to make sure your financial data is safe. It's too expensive not to.