To truly be safe from Signal’s influence you would need to audit the source code and build it yourself.
Usually I only install APK's from F-Droid, which always builds its apps from source, rather than using the developer's APK. I'm uncomfortable that Signal doesn't seem to be on F-droid, and I'm in fact hesitant to install it from anywhere else. I'm not currently set up to build Android apps myself. I'm a fairly unsophisticated Android user.