This would be fine if signal didn't populate the list with everyone who is on signal from my contacts, even if I've never sent a message. If those contacts are in a separate category ("uncontacted" or something), that's ok, but they aren't. Also the issue remains that I might need to remember someone is in there even if I've never sent a message, so I need to pin them. Plus those I actually do want/need pinned can still easily exceed 10, and certainly always exceeds 4, and always has.
For reference I only have like 35 contacts in there. So having even 20 pinned is still far from "all chats are pinned".