Only people who are bad credit risks ever come up with this take, lmao.
The sole function of credit scores is to benefit people who are reliably 'good for it' when they borrow money. Without them, everyone is treated as just as high a risk as the worst borrowers who are least likely to pay back their debts, and you gain no benefit from reliably paying back your debts. But with them, your good borrowing is kept track of, and good reputation means lenders trust you more to pay your debts back, so they're willing to lend more, and they are willing to charge less interest.
Removing credit scores changes nothing for bad borrowers, and hurts good borrowers.