Sun 18 Dec 2022 12:10PM

Adding Relay Instances

SW Sam Whited Public Seen by 191

Hi all,

I recently posted a poll about some smaller instances that invited us to join their network of relays. I was surprised to find that the poll overwhelmingly voted negative, and while I respect that this is the community position I was also surprised that Loomio polls didn't really allow for discussion, and I think the majority of the comments left were people who didn't understand what relays were (my mistake!), or focused on a specific aspect of the community that asked us to join which I mentioned off handedly, but wasn't that important. I wanted to open up discussion on the issue and see if others felt differently or if it's just me who thinks this is an important act of solidarity on the fediverse!

First, here's what the admin page says about relays:

A federation relay is an intermediary server that exchanges large volumes of public posts between servers that subscribe and publish to it. It can help small and medium servers discover content from the fediverse, which would otherwise require local users manually following other people on remote servers.

The network in question included several small, new instances mostly aligned around love of bicycles, green transport, and cities. However, the network was not specifically targeting those topics, instead it was trying to expand its timeline to give its users a broad set of interests for them to discover and they thought the values and posters on our instance would be a good fit. Someone commented in the vote that since we'd be the biggest instance we'd be increasing their database size disproportionately, and to be clear, this is true, but they invited us and are aware, the idea is that this gives their users an easier way to discover content they like.

I also saw a few comments that said they didn't necessarily want our federated timeline swamped with bicycle content, however these instances are a lot smaller than us so the converse of the previous point is also true: from our instances perspective nothing really changes since users of this instance are likely already following users from those instances and many (if not all) of the posts are likely showing up in the federated timeline already. For now, as the biggest instance, we'd mostly just be helping populate their timeline.

Another voter in the poll mentioned that they wouldn't be comfortable joining relays unless it was clear how other members are chosen, which is absolutely fair! In this case, members are chosen only if all servers on the relay vote to add the instance, so for our vote we could start a poll or find some other process.

In general I think relaying with smaller instances is a good way to help them get started. It helps users not think that mastodon is "boring" because their instance doesn't have much content on it and it helps us in our mission to support building a kinder social network. I'm not necessarily asking us to re-consider the poll right now, just for a bit of discussion: am I right here, or totally wrong? How do we feel about joining relays in the future?


Konrad Lawson Sun 18 Dec 2022 2:27PM

I suspect relays will become increasingly important as Mastodon grows and joining relays who manage membership through a democratic process built on a set of shared principles seems wise. Added to this the benefit (suggested by their invitation) to these smaller instances, this seems beneficial to the federation. I do appreciate this thread-first discussion that allows us to hash things out.

I’d be inclined to support this. It would be nice to have a few of the relevant details here or any relevant repoll.

I wonder if some relevant links could be shared here, such as the relay network details, their joining/regulation details etc., for clarity?


Scott Jenson Sun 18 Dec 2022 2:46PM

I'm surprised by the negative sentiment towards joining a relay server. It is really nothing up good for us and the concerns misunderstand what it's doing. I'm strongly in support of this.


Chris Gollmar Sun 18 Dec 2022 3:50PM

Thanks for this clear explanation! My main takeaway is that this about creating a viable community of communities that share some commonalities. I can see why people may be concerned about too much of any specific kind of content coming in from other instances in the relay, but, as I see it, the specific content is not really the point. Presumably, as the number of instances sharing relay instances grows, the diversity of users' interests and instances' focuses will also grow.


Justin du Coeur Sun 18 Dec 2022 4:07PM

The principle makes sense to me. One of the common themes of meta-discussion around Mastodon over the past couple of months (as I've been reading into it) has been that there are some diseconomies of scale: due to the effort of moderation, Mastodon tends to work best as lots of medium-sized instances, rather than a few gigantic ones like mastodon.social. But to get a medium-sized instance, you usually have to start as a small one and work through the growing pains, and that is a little easier if the overall community provides some support.

So it seems to me that providing some relaying to small instances is a larger social good, and aligns with the general mission of social.coop -- as a cooperative of instances rather than individuals in this case, but it's a similar spirit of working together to build something worthwhile.

I suspect that there is a sweet spot of how many we should be relaying with at any given time, so as not to overwhelm our own feed with external traffic, and we should probably consider de-relaying some of them as they get big enough to stand on their own (assuming de-relaying is a thing), replacing them with newer small sites.

So my sense (as, note, a newbie here but experienced at social networks generally) is that yes -- it would be appropriate for us to relay with a few experimentally, get a sense of what it's like, and then maybe set a "relay budget" in the medium term, of a rough number of small sites to relay with at any given time, with the specific sites gradually shifting in and out as they get enough critical mass individually.


Matt Noyes Sun 18 Dec 2022 4:27PM

Maybe someone explained this already, but where would toots from a relayed instance show up? On the Home timeline? The Local timeline? Ideally, there would be another timeline just for relays.

I like the idea of helping new instances get connected and of building inter-instance communication but I want to avoid the over-population of my timelines with content I haven't chosen to follow. It sort of defeats the purpose of following, favorites, boosts, etc?


Sam Whited Mon 19 Dec 2022 11:52AM

They show up on the federated timeline along with accounts that other users on your instance follow. If you're a small instance this timeline can be rather empty, so relaying others lets people discover and follow users on those instances and is very helpful for smaller instances (we'd likely be the largest one in the relay network that reached out to us, the others are tiny instances that are interested in helping their people discover users of social.coop since our interests likely align)


Giacomo Sun 18 Dec 2022 4:39PM

Initially, even I hadn't understood what relays are, so I think that the topic deserves to be treated again in the near future


Mitra Ardron Sun 18 Dec 2022 7:52PM

I've got similar questions to Matt - I find the Local or Global timelines too big to be interesting , BUT I do find the Search really restricting and pretty useless so I'd love something that expands our access to searchable stuff.


Sam Whited Mon 19 Dec 2022 11:53AM

If I understand Mastodon search correctly it only searches hash tags, not posts in general. However, relaying would mean that more posts are searched for hash tags, making the results better.


Mitra Ardron Mon 19 Dec 2022 9:09PM

Thanks Sam, that might explain why Mastodon search is so poor.


Shou Mon 19 Dec 2022 1:08AM

I think my main question is: is relaying even useful? As I understand it, posts appear in the Federated tab, which is extremely noisy and basically useless, and media/posts/likes/follows are synchronised better at the expense of higher server load/more space usage? If there's more to it I'm missing, please do tell, because I'm not seeing much benefit as it is

During the vote, I think I specified that I'm not so much against the idea of joining forces with them even if they're not specifically coop related, because they align somewhat with our values anyway. I'm happy to associate with climate change/biking/urban planning related instances, but I think we should do it in a more meaningful way.


Sam Whited Mon 19 Dec 2022 11:54AM

The space usage will be negligible for us. The other servers are mostly tiny, so the federated tab for them will be mostly empty; relaying is more of a service for them so that they can discover more users on social.coop and the other aligned instances in the relay. From our side it would also be more posts that can be searched (eg. when searching for a hash tag you'll see some of the posts, making search more useful).


Dynamic Mon 19 Dec 2022 1:48AM

Is it only the posts on those servers that would show up on our federated timeline, or also the posts on servers that they federate with? If the latter, what is the forward-looking governance plan if members of the network have disagreements over which servers to federate with?

I also wonder about network governance (or lack thereof) for behavior norms within the servers in the network. How are we affected if servers in the network have more permissive or less permissive norms than we do?


Sam Whited Mon 19 Dec 2022 11:55AM

It's just their posts, if I understand this correctly. The relay that reached out to us is more or less co-operative, one <relay instance>, one vote in terms of governance.


Dynamic Mon 19 Dec 2022 1:16PM

Thanks for this information.

Does "one relay instance, one vote" mean that social.coop would get one vote, but that network instances that have 10 members would also get one vote? What would the process of voting look like? Would members of social.coop need to vote on what social.coop's vote would be, or would we have some sort of representative who would aggregate votes and vote on our behalf?

There are a lot of social details that I think have so far been left unspecified.


Sam Whited Tue 20 Dec 2022 12:37PM

yes, each of the relays would get one vote. I don't think we really need to hash out every tiny bit of how things are going to work before we decide if we wanted to join a relay, personally. One of the existing teams could take on the work, or we could designate someone to be a representative of social.coop to the relay. It's not a huge thing with extensive governance needs, and we could always pull out later.


Dynamic Mon 19 Dec 2022 1:53AM

I think it's worth noting here that most servers are run by one person or a small handful of people who have full decision-making power to define policy. That isn't the case for us, so if policy disagreements come up between us and another member of the network, it wouldn't be a simple matter of getting the admins together in a room to hash things out; we'd need a meta-level democratic process, and I want to know what that would look like.

Item removed


Drew Harry Mon 19 Dec 2022 6:47AM

I'm still not fully understanding the implications of relaying. The wiki on the topic doesn't seem to have much detail. I think what happens when you connect to a relay is the following:

  • Toots from any member of the relay are added to our federated timeline.

  • When you search for something, it will search any instance connected to the relay as well.

And then one other wrinkle -- the other way another instance gets federated to our instance is that someone on our instance tries to follow someone on an unfederated instance. That causes our instances to federate, as well. An admin could undo that federation. But we default to allow any user to create a new federation link.

What I don't get is how a relay is different from the admins on any other instance just choosing to federate with us proactively? Is it that if you share a relay ALL toots from all member instances appear on all federated timelines, but if I follow someone on another instance only the toots from accounts I follow appear?

The whole thing seems weird to me. Our federated timeline is nearly useless to me at its current volume. I tip into it occasionally and adding some small instances is in fine with me; I doubt I'd notice.

Maybe if I were on an instance with 50 people I would find it a lot more valuable because their federated timeline would look a lot more like our local timeline. And maybe that's actually what they're asking, practically? To which I would say "this effects you a lot more than me and if you are interested, then sure!"


Sam Whited Mon 19 Dec 2022 11:57AM

If I follow someone on another instance we don't get that whole instances timeline to be searchable/in the federated timeline, just the person I followed. Relays pro-actively send all posts on that server so that they show up in the federated timeline/can be searched. For smaller instances (ie. not us, but the other ones in the relay) this makes their federated timeline actually useful for discovering new people and helps them get bootstrapped.


mike_hales Mon 19 Dec 2022 11:30AM

A 'relays timeline' like @Matt Noyes mentioned sounds good to me. I never look at the Local (social.coop) feed, let alone the Fed feed, and only sometimes at my home feed (followed folks) which is pretty diverse, when I have time to browse. Notifications/@mentions is my primary window: my Mastodon is pretty much peer-to-recognised-peer. But it would be good to have a 'fresh voices' feed, of affiliated instances/relays.

Helping small instances achieve diversity and range sounds good too.


Josh Levy Mon 19 Dec 2022 3:10PM

I think it's a great idea to allow other trusted servers to add us as a relay. From my understanding I don't think it causes any harm to us, as it just allows them access to more of the fediverse. I ran my own server for a bit and the lack of relays really hampers the experience. It's hard to find anything via search because hashtags aren't populated, you don't get insight into favorites or boosts on posts because your server doesn't know about them, your federated feed is only your follows, etc. It feels pretty isolated.


Dynamic Mon 19 Dec 2022 5:25PM

I wonder if there might be value in combining joining a relay network with switching to a fork of the Mastodon codebase that includes a privacy option for "Local" posting. The relay would make our public posts visible to more people. If this is a concern for anyone (and honestly, I don't know if it is, just trying out scenarios here) then local posting would continue to allow a posting option that's more public than "Followers Only" but less public than "Public."


Ana Ulin Tue 20 Dec 2022 4:31AM

I am supportive of the spirit of acting as a relay for new, small, aligned instances, as a way to help them get more content into their federated timelines (the impact on our own federated timeline will be negligible, as I understand it).

If/when we do want to act as a relay, in addition to the community chiming in on alignment with the instances we'd relay to, I'd like to see TWG weigh in on any additional burdens on our infrastructure, and the CWG weigh in on the suitability of the proposed instances (at least do some due diligence on #fediblock).


Zee Spencer Wed 21 Dec 2022 5:51PM

I am hesitant to offer this as a service to the community, as I would prefer there were some expectation of reciprocity. Perhaps what we want to do here is write a Relay Policy which allows Members to request Social.Coop serve as a relay, with clear guidance for how relays are added/re-evaluated/dropped.

I'm also in favor of the policy being "Relays are added only with x% of members voting, and y% approving plus a certification from the TWG that a particular relay is safe-to-add" (or an even smaller policy if it can be done)

@[email protected] Wed 21 Dec 2022 8:28PM

I would be totally FOR relay for other democratically-run instance if they're bigger than a certain number of active users (like > 10 or > 100). If not, we risk accidentally propping up structures that run counter to our goals.


Sam Whited Fri 23 Dec 2022 12:22PM

What sort of structures are you talking about here?

@[email protected] Fri 23 Dec 2022 2:30PM

Nothing extremist, but non-coop structures. If we want social.coop to survive, we have to have new people coming in to replace those that leave or become inactive, and to do that people have to know coop model is possible and can work just as good as benevolent dictator.


Sam Whited Fri 23 Dec 2022 2:33PM

Personally, I disagree. We should absolutely be encouraging the co-op model, but that doesn't mean not interacting with any other non-co-op folks. Small instances that do good things might use different models, that doesn't mean we shouldn't show solidarity.

@[email protected] Fri 23 Dec 2022 2:35PM

I was too strong on my first statement - but if the users of the other models see the coop model as viable but not for them (right now, or ever), that's fine since for them it's still a real thing, just not something they're into. I'm more running into people thinking that a coop model is a fantasy or fake.


Jeff Piestrak Fri 23 Dec 2022 8:34PM

Hi all, after being referred to this thread from the friendly admins over at social.coop, I finally took some time to digest it all within the context of my original, somewhat narrower question there:

How can we better support (hashtag) search beyond our own instance?

Here's what I've gleaned and attempted to summarize based on the above, offered in hopes it can make this easier to understand and potentially vote on the relay question again in the future...

  1. Currently, the federated timeline lists posts from people followed by members of our own instance

    From https://docs.joinmastodon.org/user/network/:

    When you browse the federated timeline, you see all public posts that the server you are on knows about. There are various ways your server may discover posts [are there other ways we have at our disposal???], but the bulk of them will be from people that other users on your server follow.

  2. Currently, hashtag searches will only return results from your local instance [and individuals followed by members of your local instance?].

  3. There seems to be very little documentation of what a relay is, and what it can and can’t do - see wiki: https://joinfediverse.wiki/Fediverse_relays - HT to @Drew Harry for sharing this above!).

  4. But according to @Sam Whited, the Mastodon admin page does say:

    A federation relay is an intermediary server that exchanges large volumes of public posts between servers that subscribe and publish to it. It can help small and medium servers discover content from the fediverse, which would otherwise require local users manually following other people on remote servers.

  5. In other words, a relay enables an instance to pull in and display (not push) posts from all members of another instance within its federated timeline (again, not just those followed by members of one’s own instance - see point #1 above).

  6. And (at least theoretically?), a relay offers the ability to search hashtags across other entire instances, not just individuals followed by members of one’s own instance.

  7. Both of these capabilities (hashtag search and full display within an instances federated timeline) COULD be reciprocal if individual instances published AND subscribed to each other’s stream via the relay.

  8. What customization can be done to control what comes in and/or goes out is still vague, and might require forking.

    For example, (if I'm understanding it correctly) @Dynamic wonders if we could have more granular privacy options that would allow our posts to be shared at an instance level only, shared with other instances via the relay, or fully public (via federation).

  9. Outstanding questions:

    a) Can a relay, perhaps operating as a kind of searchable database, also support search capabilities for individuals (not just hashtagged posts)?

    b) The Explore > Hashtags feature currently allows us to see what is trending across the fediverse (or at least those our local instance hasn’t blocked). Why can’t we also search for specific hashtagged posts across that?

    c) Can we set up a relay that allows us to only search for hashtagged posts from other instances without pulling all posts from that instance into our federated timeline? Or are there existing relays we might be able to draw upon for this search function, that don’t require us to create our own?

    d) More broadly, should the cost of setting up and maintaining relays be shared with (or born by, if not reciprocal) other relay members?

My Own Final Thoughts

After digesting all of this (or at least attempting to!), my feeling is that we should at least be open to setting up one or more relays, and/or joining others (especially if we have some control over how data gets imported, exported, filtered, or accessed). Discoverability is an important element of our core solidarity principles, and the functionalities relays offer seem to be an important tool for doing that in a well-managed and democratic way.

I also think we need to do a much better job of translating this stuff to our members. In a way that allows them to take advantage of the full affordances of the fediverse, while also be more informed participants in these discussions.


Scott McGerik Fri 23 Dec 2022 9:22PM

Your post is, to me, a good summary of the issues with relaying.

My concern with relaying is related to your 9c question. I want to prevent the pulling in of all posts, i.e. full text search, and vice versa. I'm completely okay with hashtags because we can choose to not use them. Pulling in all posts opens the door for abuse.


Sam Whited Sat 24 Dec 2022 1:11PM

a) Can a relay, perhaps operating as a kind of searchable database, also support search capabilities for individuals (not just hashtagged posts)?

Do you mean full text search? Or searching for users? Either way the relay just pulls in posts, it doesn't change anything about how search works except gives you more posts to search as if they were local. This means that users on the other server will be searchable if they've posted and their posts have been relayed to us, but full text search will not be enabled (we could do that separately, but that's another discussion).

b) The Explore > Hashtags feature currently allows us to see what is trending across the fediverse (or at least those our local instance hasn’t blocked). Why can’t we also search for specific hashtagged posts across that?

I'm not really sure what this means; you should be able to search for any hash tag and get all the posts that the server knows about whether the hash tag is trending or not? It's up to the moderators to approve hash tags for this feature, and we often forget about it, so the "trending" pages may not be very up-to-date either way (sorry about that).

c) Can we set up a relay that allows us to only search for hashtagged posts from other instances without pulling all posts from that instance into our federated timeline? Or are there existing relays we might be able to draw upon for this search function, that don’t require us to create our own?

To do this you'd have to pull in those posts, otherwise they can't be searched. With the caveat that the tech team should make this call, not me: pulling in lots of extra posts likely isn't a problem and is a trivial amount of extra space taken up in the database.

Finally, the last sentence and the next question seem to be a misconception of how relays work, I'll answer it next:

d) More broadly, should the cost of setting up and maintaining relays be shared with (or born by, if not reciprocal) other relay members?

When I say "set up a relay" we don't have to run any extra software, pay for servers, etc. this is just a configuration option that has to be set in both Mastodon instances and then they start sending posts from all users instead of just followed users. It doesn't really have any cost for us; the smaller instances might have to increase the size of their database drastically if they relay with a much larger instance, but then they get the discoverability. The instances that asked if we'd relay with them are okay bearing this cost of a larger database because of this.


Sam Whited Sat 24 Dec 2022 2:14PM

Pulling in all posts does not enable full text search, we can decide to enable/disable that separately. All it does is pull in posts to the federated timeline, nothing else changes. We also can select what servers we relay with, so it's not as if we'd do this for abusive instances, we can be selective about it.


Scott McGerik Sat 24 Dec 2022 7:09PM

Thank you for the clarification.


mike_hales Sat 24 Dec 2022 9:53AM

@Jeff Piestrak notes

Discoverability is an important element of our core solidarity principles

Yep. On this basis, yes, relays have a place in the ecosystem.

The complexity that Jeff analyses is one of the persistent troubles in the fediverse (and digital generally. And human (male) society generally 🙄 ) . . There are so many powerful tools, and so many complex interactions arising from all the tooling, that only 'wizards' can be literate in the ecosystem that results. I see the same thing occurring in federated wiki for example, where amazing capabilities invented in the past 18months (with graphs for example, or with scraped geographical data) are being leapfrogged by even more amazing capabilities (in collaborative network mapping) . . As a 'mere user' who tracks the tech conversations, I haven't a clue when, if ever, I'll find the headspace to learn to drive these powerful things.

So what? I don't know. But 'the simple user' is a basic orientation we need to somehow assert? Whatever that might mean. As regards searchability across Mastodon instances . . keep the UI simple? Keep the simple searches working well? As regards relays.. maybe add a 'guest instance' timeline?


Matt Noyes Sat 24 Dec 2022 4:02PM

Grateful to all for such a thorough examination/explanation of this idea, and to @Sam Whited for proposing it. It seems to me we are ready for a proposal, maybe using the "good enough for now, safe enough to try" standard for a limited time period (six months?).


Jeff Piestrak Sat 24 Dec 2022 7:43PM

Yes, much appreciation to @Sam Whited for the original proposal and detailed responses, and the many other contributions! I have a clearer idea of the mechanics of search and what a relay might offer. And to @mike_hales' points, hopefully we can improve our ability to translate these kinds of complexity to our members going forward (and simplify as appropriate).

Though there still seems to be some remaining concerns about the impact on our timelines, a trial period with member approved smaller instances (the focus of Sam's original proposal) seems relatively low risk while offering reasonable benefits (including the lessons all parties might learn from that).

My original interest in making a much broader range of fediverse posts searchable (by hashtag) without flooding our local or federated timelines with the actual posts would remain partly unresolved. Happy to work with others who share such an interest in framing it as a doable proposal (not sure if or how we might do this on a technical level...).


Thomas Beckett Mon 26 Dec 2022 2:03AM

How many active members do we have now? It occurs to me that if the social.coop experiment thrives that we might reach a population level - 10,000 members? 5,000? - where it feels too big and we might want to calve off a second instance. social-too.coop? Anyhow, if we did that at some point in the future, it would make good sense to maintain a relay between social.coop and it's progeny.

@[email protected] Tue 27 Dec 2022 5:07PM

Based on the comments here it seems what people are looking for is more like this fakerelay thing https://github.com/g3rv4/GMFActionDemo


tanoujin Thu 29 Dec 2022 10:39AM

Thanks to all who are trying to explain. I understood we would dominate such a relay, offering a service to small instances to help them to connect. That is fine, but I would like to see a tangible commitment to democracy from the benevolent dictators of the joining instances, and I oppose a 1 server 1 vote committee above this level of decision making. This would be the 2ct of a non-techy new member who welcomes educating texts and friendly critique where they might go wrong.