The experience of using Matrix involves a lot of sluggishness at various points in the client - waiting to decrypt messages or properly sync keys, waiting to join a room or for room search to load - these are the things that have been salient to me using multiple matrix clients with a freshly-spun-up server within the past month.
I more mindfully played a bit with my Element (web UI), and Element X (Android), and while there might something to it, and I suspect the e2e encrypted data model will always lead to some extra work required. Element seems a bit sluggish. However Element X on my Android seems butter smooth.
And event the slower Element seems far better than Discord that I'm forced to use, where I can't even scroll history without the whole thing stuttering.