They've repeatedly stated that they're working on removing the need to share your phone number with your contacts, but that's taking some time, because they want to implement it in a way that does not involve storing your entire social graph on their servers.
You'll still have to sign up with your phone number, but the only thing that can be traced back is that your phone number is registered on Signal - and only by subpoenaing Signal, I believe.