Here's the update, I've got the USB-C/HDMI adapter today. Connected it to the port that connects directly to the dGPU and even during boot Plymouth was already outputting video to the TV. I also tested hot-plugging and it just works as expected.
Now for the problems, I ran benchmarks and the performance was as expected, but frame delivery didn't look as good as when using the HDMI port on this device. It doesn't show on the performance metrics, but looking at the screen, the frametimes looked off, stuttering. I'm still figuring out where the issue might be to report it to upstream. EDIT: For people reading this in the future, I've found the issue to be in GNOME's compositor, Mutter: https://gitlab.gnome.org/GNOME/mutter/-/issues/3070#note_1865351