Should we run a Matrix server?

Hi,

We would basically act as another token for the Matrix Foundation, their Open Tech Will Save Us series being basically a nice libre software developers public discussion. I'm saying this as a bitter Nomagic user whose admin has strongly held opinions about politics and IT, but who was happy to show up and present how he used Matrix as a kitten.

In reality, the Matrix Foundation optimizes for engagement and it's a pity to come home, be greeted by cats, and ignore them to stay involved in a conversation whose real-life purpose is made secondary by the Element client's “delight” : a team paid by investors and rewarding interactions with its interface.

A saner “protocol” would be XMPP : there is about 500 XEPs (and IETF RFCs, for starters) and 4,000 MSCs. This obviously increases the cost of developing a brand new Matrix client and I only see a glimpse of light near Ruma, which provides free code (as a library) for developers. This narrows the Matrix users' client choice : some other mainstream ones don't even show modified messages properly! As a matter of fact, even if there's much more interest for Matrix, there are also fewer clients…

I know Ginny used Zulip a while ago and maybe we should try it out, but honestly I've been confused about our use of Matrix since I've come back and heard about it.

I would be in favor of trying it!

Can it be funded? Matrix servers seem expensive. Still, I am excited about this, as it seems social.coop is becoming "not just" a mastodon instance, after joining meet.coop

Other question: Are apex domains for both mastodon and matrix handles possible, or would it require a subdomain? evan:social.coop vs evan:matrix.social.coop

I know that Nomagic uses a subdomain due to LDAP (a centralized way to manage credentials, i.e. one username/password per co-owner across all services) but otherwise it's possible to serve contents on the same FQDN, so-called "domain" (but on a different port) or on the same port (but on a different FQDN).

In other words, it's possible to run a Matrix server on the same domain as a Mastodon instance because they don't use the same ports.

Cost-wise, Conduit is a small, efficient server that I expect to be ideal at our scale. The only issue in terms of server performance is Synapse, the legacy Python implementation, but it's been hard to develop something else because again, 4,000 MSCs.

I'd support it and would likely use it. I think Matrix is interesting and potentially important. This feels like a quite different thing, but complementary and in the same vein as the mastodon instance. That said I'm not sure I could contribute to maintenance myself.

Hi Flancian. Is the main benefit that we would be able to use social.coop in our IDs for branding when speaking to others on the Matrixverse? I ask, as I've not been left with the impression that we've outgrown the room we currently have on the matrix.org homeserver.

I proposed this years ago and would be very much in favor. I'd love to make Social.coop my primary Matrix instance, rather than Matrix.org. I have also had the experience of running a Matrix server (via Cloudron; we use a it for internal coordination at my lab), and it has been fairly low-maintenance.

Without diving too far into my dislike of everything Matrix, I'd like to also suggest that we consider other chat protocols. Matrix is a large, VC funded protocol (that opened a foundation after receiving criticism of the way they're run, but it's all the same people and accounts) that ignored years of community consensus building to re-invent the wheel and did a bad job of it (but had the marketing budget to convince people otherwise). Matrix is a bad chat protocol, with mostly bad clients and servers, and I would be extremely sad if social.coop used something like it instead of an IETF standard or other more community driven solution.

I use matrix a fair bit and am hugely in favour of people owning their own tech via coops. So in many ways I'd be very happy to have an account on a matrix instance thats owned & run by a co-op of which I'm a member.

However its my impression that matrix servers take a significant amount of server and sys-admin resources to keep running smoothly. I imagine we could cover the server costs and/or raise more funds as necessary - but I'm concerned about the possible burden it could place on our sys-admin volunteers.

I know a fair few matrix servers where the sys admins have struggled keeping matrix running well and spent a lot of time trying to massage it into behaving. Also a few where they gave up and closed the service.

I havent been paying as close attention last couple of years and possibly there could of been improvements that make things easier for matrix/synapse sys-admins.

I'm against rushing into this without getting a much better idea of how things are currently for synapse (I dont think, still, any of the other matrix server implementations are properly production ready?) sys-admins & ideally having a few people with the necessary skills committed to keeping the service up and running.

This is an old discussion and I haven't been active in the social.coop community over the last couple of years, so feel free to ignore this. But I'm going to chuck in my 5 cents anyway.

First, a little context. My comments here reflect my experiences as an early adopter/ power user, I'm not a developer nor a server admin. I was an XMPP champion for many years, starting back when it was called Jabber. More recently I've done a little bit of paid contracting for Snikket, a project to build a unified UX on top of existing XMPP software (so far Prosody, Conversations, and Siskin). I've also been an active user in the matrix-verse (mainly Riot/ Element but I've experimented with a few other apps) and spent a couple of years following the development of the project. So I think it's fair to say I can comment on this topic in a fairly non-partisan fashion.

From a user POV, matrix is clearly the better choice for federated chat right now. Both direct chats and group chats are easy to start in matrix apps, and there are apps for all major OS platforms (and the web) that reliably support file transfer, voice/ video chats, and E2EE (including verification of other users). Most XMPP apps are relics of the late 1990s, and their support for modern chat functions like creating groups chats, file transfer, voice/ video chats, and E2EE, is patchy, although some progress has been made on this in recent years. Snikket apps significantly improve the UX of XMPP, and may eventually be a better choice than matrix, but it's a fairly new project with limited resources, and so far it only has apps for Android/ iThings. At this stage, the Snikket server is only distributed as a Docker image, so whether that's a good choice depends on your opinion of Docker. Snikket apps (or the upstream apps they build on) can be used with any XMPP server, but this can affect the quality of the UX, depending on how the server is configured.

I've debated XMPP fans many times in the fediverse on the topic of matrix, and most of their criticisms boil down to the server performance of Synapse. It's true that Disroot and some other community-hosting organizations switched from matrix to XMPP for chat, because of poor server performance. But the matrix folks openly acknowledge that Synapse, as a prototype, has significant downsides. It's also true that its performance has improved significantly over the last couple of years since Disroot made that decision.

So the question for social.coop is what's more important, making live easier for users, or for server admins? If it's users, the best choice, for now, is probably matrix. If it's admins, the best choice is might be XMPP. One option could be to stand up one of each for a limited testing period, so the social.coop community can evaluate both, from both user and admin perspectives, and then make a final decision on which one to use as an ongoing service.

Thanks for your perspective, @Danyl Strype. This is really helpful context for me. I know MayFirst.coop, which has stalwartly stuck with XMPP, is now exploring Matrix finally as well.

FWIW, I have been running Synapse on my lab's server for a few years with no problems, maintained with Cloudron. But that is a smaller user-base.

It's also telling that there already is a Social.coop conversation happening on Matrix, using a Matrix.org space (which is featured on the main info page of our Loomio space). Clearly people in this community find utility in it.

Given this conversation about extra funds, I wonder if folks from the Tech Working Group could ballpark a sense of what additional funds would be needed to support a Matrix server. I'm not actually sure who is a member these days...

I just wrote a load of stuff related to this in the social coop tech chat room on matrix

Ill spare everyone me reposing it here.

Can read it over there at

https://matrix.to/#/!akkaZImONyQWKswVdt:matrix.org/$1665444472150394ZaVfj:matrix.org?via=matrix.org&via=autonomic.zone&via=pub.solar

Or

#socialcoop-tech:matrix.org

It took me some effort to find your comment after clicking that matrix.to link. A summary of your key points here would be helpful if you have time to post one.

An eJabberD server might be a good option, once the matrix protocol support is added to the free code Community Edition:

https://www.process-one.net/blog/matrix-protocol-added-to-ejabberd/

I don't know others, but for me would be helpful to evaluate this proposal what is it's objective. Is it only to have a chat base for our community or are there other things too? If other, what other objectives we (can) have? Are we pointing to provide what services, if that is knowable/discussable now? For example, could we provide some private groups too? What about interoperatibility, or future migrations?

Sorry if these questions seems obvious to all of you, but since I'm new and this is a bit out of my waters, this could be a way to help to help. :)

Best...

I defer to others that are more intimately aware of the tech and politics around Matrix, and sorry if this is off-topic but I'm curious if anyone can comment on why Matrix (or maybe more specifically synapse) is so costly and difficult to maintain?

Re: purpose - different mediums have different purposes and support different kinds of interactions. Eg. we are here as opposed to masto because it's easier to have longform, structured, discoverable conversations across the whole instance. I see a chatroom as having non-overlapping purpose with the masto instance and here as a more ephemeral way of communicating within instance. I think one think we can do better as a co-op is intra-coop cohesion, and currently I don't know of a good way to shoot the shit with y'all. Loomio feels very formal, seeing as how it is the governance mechanism and all, and there has been limited uptake in the guppe group.

I don't have a strong preference of any particular implementation over another, but I also would like to be able to use this as my home instance, particularly if it's possible for me to start my own "rooms" eg. for projects that I'm a part of where I can invite some collaborators to talk without needing to use eg. gitter or start my own server. From what I understand about Matrix/Element that should be possible (question mark?) but also thinking more broadly about our engagement with the larger decentralized social world I think it would be lovely to show up and build more of a cohesive community together :)

Yes, I think you should be able to do that in a Matrix instance we run—create your own spaces. But that does raise the question of purpose. Is the goal to provision Matrix for Social.coop members to use in their lives in whatever ways? (In which case, why not just use Matrix.org, which is free?) Or is it just for Social.coop business?

As much as I love Matrix, I guess I'm starting to lean against this.

this, to me is a pretty good reason, in line with general decentralization principles :)

@Nathan Schneider Do we use the social.coop Mastodon instance only for social.coop business? I think most of use it in lots of ways that aren't directly related to the ongoing work of social.coop?

If there were a social.coop Matrix server I would likely want to use it to connect to all the channels I'm currently in (not just in social.coop space). But maybe we would want to have some rules around the types of channels that could be created and hosted directly at social.coop and who could create them?

This proposal also has some overlap with the current proposal to join MayFirst as they already host chat services. It may be worth considering whether it's worth having the duplication with MayFirst if we do in fact become an organizational member. More info: https://www.loomio.com/p/Cr9mG76W

May First does not offer Matrix (though it might in the future), so I don't think it's duplication.

I said "some overlap", I think that's accurate and it's worth considering using their chat services in the future instead.

I'm generally in favor of exploring a matrix or other federated chat system with E2EE support via social.coop.

My concerns about us having our own matrix server are mainly around the resources involved. Unless theres been significant changes in the last couple of years it will take a lot more money and, I think, more importantly attention from our volunteer sys admins to keep it running well. We havent always done amazingly keeping our mastodon instance running well and updated. Theres been a number of people whove put in heroic efforts over the years keeping things running.

To answer @jonny appatrently the problem with synapse is largely that it's written in python which isn't at all suited for the task at hand.

The folks at matrix.org have been working on an alternative, Dendrite (i forget the language) for years, but its still missing significant features. Same with the independent server implementation conduit which is written in rust.

Id agree that the point of running our own server instead of using matrix.org would be similar to why we have our mastodon rather than all just use mastodon.social

Finally an alternative is to use XMPP, which is functionally similar. We could do this more easily. Its been mentioned above that one of the two main server options eJabber is in the process of rolling out support for the matrix protocol, which is interesting.

Also as Sam has been suggesting May First run a XMPP server for their members, so we could maybe make some arrangements with them? Provide their members with use of mastodon if ours can use their XMPP?

I regularly use both. Matrix feels more new and vibrant with more people in its public rooms. Also has more bugs and federation issues.

XMPP feels more basic/old but has been pretty rock solid for me. The main android client works well and is likely to get a major update soon(ish). Also has amazingly low battery drain on phones without Google Play Services, unlike matrix/element.

Not sure if theres a decent iOS xmpp app yet? I've heard people grumble a fair bit, but dont really pay attention.

Lots of app options on desktop / laptop

My concerns about us having our own matrix server are mainly around the resources involved.

Agreed. The Synapse server is difficult to scale efficiently.

That being said, both the official Go rewrite (https://github.com/matrix-org/dendrite) and the third party Rust one (conduit.rs) have come a long way. They may be easier from a technical operations perspective even if they haven't reached perfect feature parity (yet -- but doesn't seem far off).

Regarding XMPP, there is a current integration going on between a server and clients for Android + iOS in https://snikket.org/

I was on the XMPP Board one year almost 20 years ago now. I’m also in favour of Matrix.

As a protocol and community it is doing a number of different things. The code and protocol are open and able to be used by anyone.

here’s a metaverse project being built on Matrix https://thirdroom.io

Real-time encrypted group chat is a different, complementary thing to Microblogging on ActivityPub.

I’m in favour of running a Matrix instance.

I strongly support the creation of a Matrix server for social.coop. I am currently on a private (unfederated) Matrix server that my spouse (Tim) runs off of a home computer, and the interface is very comfortable and easy to use. The interface is similar to Discord or Slack.

My partner, Tim says:

"Our server runs on Dendrite, which is the official/future higher-performance replacement for Synapse—although the protocol continues to evolve, so Matrix developers are having to update *both* servers, dividing their efforts. It uses Postgres for a database but is otherwise fairly self-contained. I have it installed via a set of Ansible scripts <https://github.com/timmc/commapps/blob/master/roles/matrix/README.md>. Ours is not federated, which is partly for performance reasons and partly for safety reasons. (Our young kid is on it, and there's no way to block DMs from off-server, I think?) Dendrite has improved dramatically in stability and performance recently, so I'm optimistic that it's a good path to take. (Joining a huge room also might no longer bog down or kill the server these days, but I haven't checked. I'm also running this on a 10 year old laptop, to be fair.) We also have our own Element-web instance, which is extremely easy to host—this is important for the signin experience, as the main Element instance defaults to signin to matrix.org. And finally we run a matrix-registration UI, which allows friends to sign up using a registration token. Social.coop would probably create accounts manually instead, though."

Tim also notes that we have run into some glitchiness in the user interface for encrypted messaging, and that we mostly only use encryption for 1:1 chat, not for rooms.

He says he's found it easy to administer (though federation might make this more complicated), and would be happy to answer any questions that others might have.