I personally really like Matrix, but there are a few outstanding complaints about it. The biggest one is that the reference implementation everyone uses by default is known to be bloated and slow, and poor at scaling. Server admins have had a huge challenge of supporting a large amount of data for things like room history, which in the past required propagation to every server hosting every participant. The protocol itself has been described by some developers as overtly complex.

Some of this seems to be improving, particularly with development of a Go-based backend implementation, Dendrite.