Sat 15 Jun 2013 10:17PM

Add support for groups in diaspora

N Nick Public Seen by 147

Groups are a hugely important feature for many who might think about moving to diaspora. As such, they are a huge priority.

However, there are some technical difficulties - how should we move forward with these?


Tom Scott Sat 15 Jun 2013 10:24PM

The technical difficulties amount to federating messages. By creating a groups feature, we are essentially extending the project to also be a large distributed forum. Similar to newsgroups, I assume, people would subscribe to different groups and receive all updates on those channels. If we want to model groups after Facebook, we'll need to federate group comments and group messages separately. We can also model groups more like Usenet and simply have 1 mode of transport, leaving the hierarchy of messages up to the view layer and disallowing commenters to alter the subject of their federated group message. Unlike sharing posts, which federate to a whitelisted select few people, group messages would be federating to a much wider group of people. I would raise that as a concern just as a gut reaction, but I'm not sure of the actual impact as I do not run a large pod.


Nick Sat 15 Jun 2013 10:34PM

@tomscott - thanks - I'm not sure I fully understand; what does federating messages (as opposed to posts?) involve?

My initial reaction is that group messaging is less important than group posts etc. Or could group messaging be implemented later - client-side (?) could take the list of group members and add it to a private message to: list [I guess this wouldn't currently work as you have to have mutual sharing to be allowed to pm someone].


Nick Sat 15 Jun 2013 10:35PM

In the longer term I would also be in favour of having plugins for groups to be able to have their own wikis or etherpads for collaborative editing...


Tom Scott Sun 16 Jun 2013 1:30AM

@nickdowson well, it's more a matter of volume. if i post in a group that has 1000 subscribers, that message has to get federated out to a maximum of 1000 pods, quickly. that is a different scaling problem than, say, Facebook has, because they can just spin up a bunch of web servers that all connect to the same database cluster. with us, we're attempting to keep multiple databases in sync to ensure the data is distributed to each necessary outfit. by "group messaging", i meant group posts, by the way…i'm just using the term "messaging" here to describe how pods communicate. sorry if my language is a bit unnecessarily esoteric. :)

anyway, my major concern is how are we going to distribute this data quickly and efficiently across a wide range of pods. in the current federation protocol, we do have some issues wherein a post won't federate completely across everywhere it should be federating. i don't believe there's a fault-tolerance system baked in that ensures posts always make it to their destination. these features seem NECESSARY for group-based interaction, because suppose you make a post in a group and 50/100 of the pods that need to get it, don't. that discussion either dies, or worse, it gets replied to. the replies federate across the group and are referencing a post that for half of the pods doesn't even exist. before we implement a group feature, we need to figure out a solid solution to that problem.


Sean Tilley Sun 16 Jun 2013 3:29AM

So, one of the things groups currently would depend on is federating properly. We also need to think of a basic UX implementation.

Pistos had a groups feature on his fork of Diaspora that I really loved. The problem is that his feature is not compatible with Backbone.JS, and so would probably need to be rewritten, with tests.

What I liked particularly about Pistos' version of groups was that it was similar to how StatusNet implemented groups. Groups could be mentioned in a post using a bangtag, so !cats@joindiaspora.com would go to a cat group hosted by joindiaspora. Joining a group is kind of like a followed hashtag, except that there's an admin that can edit the page, or remove entries if they're offensive/spam/whatever.


Sean Tilley Sun 16 Jun 2013 3:34AM

The real barrier, I think, is federation support. At some point in the future, we may need to further consider our federation system and think about going with a better-supported protocol, such as OStatus as a whole (Diaspora only uses some of OStatus), or Tent, both of which are exploring solutions to existing problems that we ourselves have seen in Diaspora and other networks.

One of the long-standing problems of our federation system is that it currently needs refactoring. A general consensus is that putting federation into a layer of some sort may be beneficial, and even thinking about touching protocol work hinges on the state of our federation system and workers.


goob Sun 16 Jun 2013 9:42AM

On that last point, @seantilleycommunit , would it be worth doing a call for volunteer developers to form a working party to look into the feasibility of putting federation code into its own layer, and implementing it if feasible? It should be headed up by one of our main current developers, but could include some people new to the project. It seems to be a high priority, but the size of the task seems to be holding up getting it done. I think it might be a good idea to try to form dedicated groups for some of the biggest and most important development tasks such as this and creating an API. What do you think?

That's really a separate discussion, but I moot the idea here and if you think it's a good plan, I can start a separate discussion.


Nick Sun 16 Jun 2013 7:19PM

Ok, thanks for running through some of the difficulties. @goob that sounds like a good idea - I would think that going with a well-supported protocol should also be high priority; do the federation layer and the protocol support need to be addressed at the same time or from a technical perspective does it matter in what order they are addressed?
@seantilleycommunit - as I understand it Ostatus is not privacy aware? correct me if I'm wrong! I would think that developing/supporting/collaborating on a protocol that goes wider than diaspora is highly important to the success of both diaspora and the social web (perhaps this page is not the place for that particular discussion though)


Rasmus Fuhse Sun 16 Jun 2013 8:53PM

Great discussion here. I fully agree with Sean, but also want us to keep in mind, that groups in Diaspora* should probably something more than groups in OStatus. Groups in OStatus are like an open channel to which everyone could post messages to and anyone could subscribe that channel just like following a person. But when I think of groups, I'd really want a group to not always be public and give always public information to everyone. The group should especially control, who can join the group (admission-mode) and who can read which of the group's contents. So to me it seems as if we could take the groups from OStatus, but also need to add a privacy-layer.

Nick Dawson, adding further information to a group like etherpads, wikis or file-repositories isn't the very big deal after all. Since we rely on the activitystreams-protocol, we can support any activities for any modules in the future. So when Tom Scott says "message", he most likely means a generic message, that could be anything like a posting, a like or an editing of a wiki-page.


Nick Sun 16 Jun 2013 9:29PM

@rasmusfuhse I agree about the importance of privacy options (though as a start I think a blanket option to make all content private/public is fine).

Could you tell me more about the activity streams protocol?


Rasmus Fuhse Tue 18 Jun 2013 12:04PM

Activitystrea.ms ist the webpage you're looking for. An activity is a small piece of XML or JSON, that consists of at least subject, verb and object. A casual posting is the triple of "Rasmus", "post" and "bla bla". But you could also deliver any activities like for example "Rasmus", "edit", "My Wikipage".


Nick Wed 28 Aug 2013 12:50PM

Just bringing this up again while support for groups is hampered by federation, I would propose a halfway house:

We implement groups, but restrict them to only users of a certain pod - and make it clear that this will be the case until future development makes federation possible.

I think this is important as a proof of concept and intent, and will also make diaspora potentially highly useful to a range of organisations and social movements - who could host their own pods and groups for decision making and communicating by members.

This would also be great for growing the diaspora network, as organisations are likely to have the resources to set up and maintain their own pods.

(This discussion is also related to this one here: https://www.loomio.org/discussions/1188)


Tom Scott Wed 28 Aug 2013 5:31PM

@nickdowson By forking diaspora/diaspora, you (or someone else) can spike on this right now. There aren't many roadblocks when getting Diaspora to run off a different Git branch, so by all means...please do some experimentation and tell us what you find!


Nick Wed 28 Aug 2013 6:28PM

alas @tomscott , if only I could code!


Sean Tilley Wed 28 Aug 2013 6:33PM

@nickdowson That's an interesting thought; at the very least, we could have some basic code in place that could be improved upon later.

I think a good first step, though, is getting @florianstaudacher's Diaspora federation gem working with Diaspora. That could allow us to abstract away the working federation code in D*, making federated feature development much easier.

Actually, doing so could bring two very good benefits:

  1. Allow for a modular federation abstraction. So, let's say we fork the existing Diaspora federation gem, and do one for Friendica Red's Zot2 schema, or we implement the Tent federation API. To some degree, we can make switching out gems easy, so that we can experiment with different protocol standards and see which ones suit our needs best.

  2. With a level of abstraction, all of our existing federated features will make calls to equivalent code in the gem. So, swapping out federation gems would hopefully mean that features would have to only be adjusted very minimally.


goob Wed 28 Aug 2013 7:51PM

I'm not keen on implementing a feature that would only be available to users within a pod and not cross-network. We need to be doing everything we can to enable and encourage inter-pod communication and mobility. Having a feature which only work intra-pod would mean that people who were connected would feel they need to be on the same pod as each other, which would lead to people wanting to migrate towards the huge pods, where all the people are, rather than away from them.

I appreciate your desire to have groups in Diaspora, Nick, but it can only be of use to individuals, and to the network as a whole, if some fundamental things are in place first. Let's focus attention on getting federation working properly, so that being on a single-user pod gives pretty much the same experience as being on a huge pod, and worry about features like this once the network architecture is ready for them.


Tom Scott Wed 28 Aug 2013 7:55PM

@seantilleycommunit do you know where I can find that gem so I can help him? This has been something I've wanted to do for a long time and had no idea work was already underway.


goob Wed 28 Aug 2013 9:18PM

@tomscott , there's a link on this thread: https://www.loomio.org/discussions/3704


Rasmus Fuhse Thu 29 Aug 2013 7:45AM

Since groups need a special way of federation information to other pods, we couldn't program the federated groups unless we have put federation into an abstraction layer. It would be no good to make ourselves double work.

But a prototype could be nice. Pistos already did something like this prototype two years ago.


[deactivated account] Sat 8 Nov 2014 5:44PM

Let's keep this thing going! The last post was 1 year ago!? Why cease making Diaspora more awesome?


Brent Bartlett Sat 8 Nov 2014 8:13PM

@riderplus Because somebody has to code the darn thing, and Diaspora doesn't have enough developers?


[deactivated account] Sat 15 Nov 2014 6:13PM

well, @brentbartlett, complaining doesn't help anyone, does it??


goob Sun 16 Nov 2014 1:04PM

Yet you did it in your previous post...


[deactivated account] Sun 16 Nov 2014 4:01PM

I fail to see where I complained, but yeah, Goob, with your conservative views on how things should look like, it's probably really difficult that anything WILL be improved around here...


goob Sun 16 Nov 2014 7:04PM

Ah, yes, always good to slip a personal insult in there, well done.


[deactivated account] Sun 16 Nov 2014 9:40PM

You may take it as a compliment, I called you a "conservative", it's not necessarily a bad thing. But you are insulting people with your attitude, just so you know. I read your comments - you like being a "killjoy" most of the time. If you find that useful and "open source"...I don't. Sorry, but I can't admire people that most of the time come with "impossible", "not worthy", "useless". Maybe you'd realize: that's your way of seeing Diaspora*'s progress...and that's sad, isn't it?


[deactivated account] Sun 16 Nov 2014 10:26PM

Why so serious ? O__O


Jason Robinson Mon 17 Nov 2014 8:22AM

Chill please..


Matt Sat 4 Apr 2015 8:58AM

Does anyone know if this is still likely, e.g. if federated messaging is moving forward?

Our friends will only switch to diaspora if there are groups! :/

Thanks for your work..


goob Mon 6 Apr 2015 6:45PM

It'll happen one day, when someone steps forward to create it.

If you really really want this feature, you could consider adding a bounty on it! https://www.bountysource.com/teams/diaspora


Ivor Stodolsky Tue 17 May 2016 10:04PM

Any news on this?


[deactivated account] Tue 31 May 2016 10:43AM

How are aspects different from groups ? or would having groups mean people don't need to use specific #tags. ?.