I don't have quite the same issues, but I am fairly regularly running into issues where software on the other device is asking to play audio, and stealing focus. Make sure to mute unpredictable sources of sound (like Windows sounds or phone notifications).

Bluetooth devices and/or software implementations of audio are incredibly buggy.

Examples:

• Saying "OK Google" does this, and sometimes detects other sounds. I have to try to never say Google if I'm on Discord for example.

• Google Photos: the memories have audio, and will steal focus even if the audio is off.

• Spotify will sometimes get in a mode where it will constantly lose focus, even though nothing else appears to be demanding audio. When this is happening playback in other apps on my phone will continue to work as expected. This only appears to occur when my computer is on and connected to the headset via Bluetooth.

Other Bluetooth issues:

• Old games may not reliably play audio over Bluetooth. Fallout New Vegas would regularly get into a state where sound effects & voice wouldn't play, but the radio might continue. Plugging in speakers and using those for New Vegas worked, but introduced other buggy behavior.

• Calls may request exclusive access to the Bluetooth device. Like if you're on Discord, game audio may fail to play, rather than just playing in a degraded quality, or some other more graceful failover. This may be fixed, I've had it work at least once with other headphones, but haven't tested it further. I just have an external microphone now, and have disabled the "headset" functionality in Windows.

Multi-point may solve your issue, but be aware that what you're asking for may introduce a bunch of new problems. I still use Bluetooth headphones frequently, and in the way OP seems to want, but there are still significant growing pains, and I'm not sure I can recommend it.