It's not that as bad as it seems. The way it is set up is actually very privacy-minded with the use of cryptography. I would encourage you to read more about this.
The primary safety concern is stalking. For example when a stalker puts an AirTag on your belongings or vehicle. iPhones can detect unknown airtags moving with you. Android devices should gain this capability in the future.