Makes it a lot of sense, and P2P is exactly what we're discussing here. But P2P can take a lot of shapes, two of those shapes are "federated" and "distributed".

My issue is not that Matrix is moving towards P2P, but this particular user saying "unless everyone is running a server it's not federation" which is strictly false.