I've found that Conduit handles it a lot better than Synapse does.

Tbf, they actually are. Element X and the sliding sync are ready I think. If the server and client both use it, I think you can use it today. Or maybe the whole matrix 2.0 where that's all integrated isn't pushed into main/master yet. Im any case, I read it was ready maybe a few weeks ago.

Eh, that's a bit of an exaggeration. There is a lot of room for improvement, but it works pretty well today, especially if your rooms are relatively small (e.g. a few dozen people) and you don't use an overcrowded instance (e.g. not the main Matrix.org one).

If you can host your own, you can adjust the resources so it works well for you. If you can't, just avoid the main instance.