Creating self/community-hosted replacement(s) for GitHub

"This announcement seems to have been the straw that broke the camel’s back for a lot of people" indeed. Here's another alternative for hosting git repositories: Phabricator. It is an Apache licensed online collaboration platform, that includes software repositories such as git, mercurial and a range of practical collaboration tools, to share passwords, make blogs, Q&As, Wikis, documents, calendars, timetracking etc. At the CommonsCloud project we have chosen this platform not even in the first place of the git repository integration, but particularly for the welldesigned selfmanagement approach. One can create groups and subgroups to emulate complex organisations while controling the visibility, editing and join rights for each object, without even relying on admins external to your project. This platform is used in WikiMedia Foundation, KDE, Blender, Apertus and several other, large communities. Let this move of GitHub be the last straw to move away to community controlled platforms! (BTW here more about the making of CommonsCloud).

@wouter how or where can I learn more about that OdooCoop project? [edit] Found http://odoocoop.cl/ - is that it?

Might be similar to something that Coopaname was working on. I don't think they ever finished, but they had a start. https://bitbucket.org/ccomb/rea/ and a demo https://odoorea.demo.prelab.fr/web/login

Hi Bob, OdooCoop is another project of the femProcomuns multistaekholder coop, where we also coordinate thh CommonsCloud Alliance. So the OdooCoop is an alias for "Odoo for Cooperatives" (we have no aspirations of licensing the word Odoo from the Belgium SA) and is an alliance of cooperatives in and around BCN that are co-developing, sharing costs and making replicable an Odoo-based ERP for the social and solidarity economy. You can find more info in the issues and wiki at our account at gitlab: https://gitlab.com/femprocomuns/odoo-coop/issues. We are also in contact with Valeureux (Sybille and others) who are working on the Coopaname case. Theirs is an important work to adapt Odoo, such that they have called it WeZer. So far we are learning the basic development and functional skills with a growing dev team of some 6 people between the different entities. Ultimately these efforts and yours could combine. But we have a priority to get our MVP ready of what we call "project0", that is the minimum platform to self-manage our cooperative. That ranges from keeping track of co-owners (that any cooperative needs to do for each typology of ownership and personal contribution levels), managing analytic accounting for different business units, what we call "Groups of Cooperativised Activity" (groups such as OdooCoop itself, or CommonsCloud, or our IoT community network The Things Network Catalonia to name a few), to manage SEPA debit instructions and synch them with banks (or other payment networks, but that's for MVP2).
Well, these requirements may seem very basic, but are the minimum for us to get working decently, and for other coops they are an interesting opportunity to make their operations much more efficient and self-managed (with real time data, think that most small orgs have economic insights after the end of each quarter...) Well, of course you knew all that already. I know that you guys are working on very interesting value accounting propositions. The thing is that we should get our MVP out or our fragile economy collapses. You can imagine more and more people joining the collective project and in need to make ends meet, while we're still with spreadsheets ... But that shouldn't refrain you from criticising us. Please comment...

I wouldn't dream of criticizing. Have you talked to Christophe at Coopaname? He's the person who was working on an Odoo project.

Also, the Mutual Aid Network has deployed a system based on Wezer and I know are looking for collaborators.

One advantage of what Christophe was doing is it's based on REA which is an appropriate accounting and economic model for networks, as opposed to single business enterprises. But I don't know how far he got and whether he still wants to work on it.

Anyway, those would be two different potential directions for collaboration: Mutual Aid Network for Wezer, or maybe Coopaname for Odoo/REA.

Bob, is that Christophe Cessetti or who is it? Will be good to be in contact. I know from Sybille that Jullien Dussart was also working on the Coopaname project IIRC (that was more than a year ago). So time to check again how their work is progressing.

Christophe Combelles

@wouter I love the piece about the technical choices for CommonsCloud and I'd really love to learn more about the reasons for going with Phabricator over GitLab?

The Gnome project evaluated GitLab and Phabricator and in the end decided GitLab was best for their scenario.

Why is Phabricator better than GitLab for the CommonsCloud scenario? (I note the Gnome's evaluation doesn't mention e.g. the rudimentary slack-clone that is part of Phabricator - also seems potentially easier to be adapted/ used for non-software projects)

Time flies. Thanks for the question, Josef, about GitLab vs Phabricator. Let me first state that I haven't seen great benchmarks, i,e, profound, well documented comparisons between the various free software tools. Ours is mostly based on direct experience and that of relevant communities.
In the case of Phabricator, there are several global communities who run their instance, two that we are in direct contact with: Wikimedia and KDE. Phabricator seems to be very apropriate for supporting the software development workflow, which for the KDE community is obviously key. In Wikimedia I think the use is wider than that. https://phabricator.wikimedia.org/
For us, we were looking for a platform that could adapt to the needs of diverse community oriented teams, not only of software developers. To be honest, we have been running Phabricator now for six months, and we haven't enabled the software repos and other sw dev tools yet. And still it has been already very useful for different teams and communities. Phabricator allows you to define projects and subprojects as you like and manage tasks, use kanban workboards, share passwords, define view/edit/join policies in each task and (sub)project in such way that users can selfmanage. There's a few dozens of other applications integrated in it, such as wikis, blogs, badges, mockup designs, timetracker and we're seeing how we extend it.
There's a common line tool for the admins; and there's arcanist, a command line tool for software development, https://secure.phabricator.com/book/phabricator/article/arcanist/
WRT parallel objects and merge: Phabricator can do this in the wiki, and in the tasks, besides obviously in git for software repos. Se can have a task with parent and child tasks which are in some level coming back. The user interface intuitively depicts these relationships graphically.
BTW, if you're interested, we are working with a docker container with Phabricator in it for quick replication and deployment. And we are working on the translations to make it truely multilingual. A major challenge that we see is how to integrate Phabricator file storage with NextCloud; we haven't figured it out, but that would be a major step forward IMHO. https://projectes.commonscloud.coop/

@strypey

I further propose that all apps who currently do their dev on GH consider moving it to the same instances,

We're working an ActivityPub-based OAE project and are watching https://github.com/git-federation/gitpub
If they can get it going, we will move there, because we would like to have all our economic activities and social interactions interconnected in one fediverse. We think that could be the start of a federated economic system.

If GitPub doesn't happen,, Phabricator via CommonsCloud looks pretty good.

@ivan116 has written the first draft of an overview of our OAE project, called OCE (Open Cooperative Ecosystem). The first app is called Agent, because it is an agent-oriented architecture and Agent is your agent in the fediverse.

I further propose that all apps who currently do their dev on GH consider moving it to the same instances, so we can all continue working together easily

Really good thought. Will stay tuned to the discussion.

One to consider: https://git.coop

Here's another alternative for hosting git repositories: Phabricator.

Thanks for the tip @wouter . My understanding is that this is also in the same category of project management platforms as RedMine, which the pioneering BetterMeans was a fork of. It looks like RM can now host its own Git repos with this plug-in:
http://redmine-git-hosting.io/

One thing I like about GitLab is that despite a very noob-friendly web UI, everything is a Git repo under the hood, including Issue tickets and documentation wikis. This means power users can track and contribute to everything from the command line, edit conflicts on wiki pages can be solved with Git merge, and backups / exports/ imports are simple Git pull/ Git push operations. Can Phabricator do this too?

@bobhaugen

We're working an ActivityPub-based OAE project

What's it called? I'm keeping a list here (for now, soon to be moved to the wiki on that project):
https://gitlab.com/fediverse/fediverse.gitlab.io/issues/8

watching https://github.com/git-federation/gitpub

I saw that. Cool concept, and timely ;) For anyone new to GitPub, it's a proposed standard that extends the ActivityPub standard (used by Mastodon, Hubzilla, PeerTube ec), in a way that could allow federation between community-hosted code forges, whether they run on GitLab, Gogs, Gitea, Phabricator, RedMine etc. Work on plug-ins for each widely used platform need to be in progress as the spec is drafted, to keep the draft focused on getting things working ASAHP.

@graham2

One to consider: https://git.coop

Ae, this is a GL instance run by an existing coop. Groups can get access by buying a share in the coop (share price negotiable, minimum $1), not sure what membership criteria are though. Social.coop is looking at buying in and using it.

@strypey

We're working an ActivityPub-based OAE project

What's it called?

https://docs.opencoopecosystem.net/

I don't think that doc says anything about ActivityPub yet, though. We just decided on that as our substrate, after looking a lot at SSB and some at Holochain. And it's tentative, we're pretty sure it will work, but don't have it up and running yet.

Here's @ivan116 's doc about the first app, called Agent. It will be an agent-centric architecture.

Sweet as, I'm just making a list of projects to watch. Once they're production-ready, and can interoperate successfully with other AP apps they get added here:
http://fediverse.party/

One to consider: https://git.coop

Ae, this is a GL instance run by an existing coop. Groups can get access by buying a share in the coop (share price negotiable, minimum $1), not sure what membership criteria are though.

You just need to agree to the rules and complete a membership form and buy some shares.

Apologies if I missed this, but does anyone know if any of these options can support federated instances? Even better if there is any movement towards being able to federate instances of different git based software so we an re-create the network effect that brought many of us to github?

There are folks working on two different approaches to federating software forges. One is ForgeFed (formerly GitPub), who are working on a set of extensions to the ActivityPub standard for this. The other is Drew DeVault, who is working on a system using Git and email protocols (see his test bed at sr.ht). I was initially really excited by the idea of using my fediverse account to follow a software repo, or comment on an issue, and having @mentions from that repo come up in my fediverse app. But Drew makes a good case for using the email-based tools that are already built into Git.

Maybe the answer is some combination of both? After all, Git has its own set of protocols, email makes use of a number of protocols (POP, IMAP, SMTP etc), and ActivityPub is a standard set that includes a number of protocols (ActivitySteams, JSON, some AP apps use Webfinger for @mentions etc). Hopefully the ForgeFed folks take Drew’s comments on board, and come up with a standard set of protocols that together provide everything we need to federate software forges, while being as easy to implement as possible.

not the right place I know, but just wanted to express my gratitude at being able to meet you, even if we talked way to little, but great to have seen you in Hong Kong

It's great to have met both of you this year for the first time in real life, separately, so it gives me pleasure to know that you have also met each other! Face-to-face just adds that extra depth, doesn't it!

We are running a community base gitlab on cloudron at https://gitlab.divvydao.net. Installation was a breeze. We have not enabled any federated login yet. It seems to use a lot of system resources and I would not put a lot of projects on one instance. I worry that a project becoming too successful will overload the system. We are using it for projects not ready due to security concerns etc to be put publically on github later. The intention is to publish on github when we go public.

This is quite interesting: https://superuser.com/questions/1162907/setting-up-an-encrypted-git-repository
I have a very strong desire to move away from github, since the Microsoft fiasco. As usual.

Will keep an eye on this space

Sure, but setting up a private Git repo is not the problem. That's easy
if you are comfortable using Git on the command line. You can do it on
any computer you can install Git on. There are ways to allow certain
people to pull and push from a private repo, and you can also make it
public. That's not the service that makes GH unique.

Neither is providing a web interface for interacting with Git repos.
There are various free code packages for hosting a web front-end for
Git, including GitLab, Gogs, Gitea, Phabricator, and possibly others.
Many organizations and projects now use one of these for their own
community-hosted web front-end for Git:
https://wiki.p2pfoundation.net/List_of_Community-Hosted_GitLab_Instances

What makes GitHub unique is simply it's network effect. It's a place where
there are many, many projects, and many, many developers have accounts
there. This is what both ForgeFed and Sr.ht are attempt to replace, by
providing a common protocol that can be used to connect up community
hosted Git forges,

Well, thats very thorough, thank you.

My operating system has basic version control built into the user-interface layer, but the perspective is slightly different, because it operates at the point of decision, rather than at the point of storage.

still, I expect we will use git as it is excellent.

I assume in the last paragraph you are referring to the NetworkEffect of GitHub. Yes, we need to build a bilateral gateway between github and whatever other tools we intend to migrate to. This is the way to deal with the network effect in any scenario

Be prepared to have your predictions come true

ᐧ

Please note that Cloudron cannot be considered Free software, since they tie their users to a central organisation.

• https://github.com/libresh/libre.sh/issues/144#issuecomment-367701427

We're also using GitLab, also for project management via issues and boards, and offer an instance at lab.allmende.io. Other known instances are framagit.org, git.indie.host, next to others.

Trivia: Since Redmine has been mentionned here, and people are also asking about nice group organisation interfaces, I can only suggest to have a look at https://taiga.io, which started out with the code name "Greenmine" ;)

@jonrichter

Please note that Cloudron cannot be considered Free software, since they tie their users to a central organisation.

This is exactly the sort of discussion that this group is useful for. Can you supply more details about this? The GH link you provided here gives a links to another thread in this group, which in turn features this link:
https://cloudron.io/documentation/installation/#cloudron-store-setup

On the face of it, the situation seems similar to Docker and DockerHub. The software itself is free code, published under an FSF-approved license. But as you say, the default install is set up to use a non-free network service, that can be used to install proprietary add-ons. Docker can be used without DockerHub and still be useful. Is this the case with Cloudron, or is it useless without the non-free app store?

Last year we've been running some Cloudron instances, and at first it matched very well with the ideas we had with the #CommonsCloud project, in that it provides a unified user account and a catalogue of free software webapps. Their setup makes maintenance really easy, and you can have their remote update service for a low cost. I have been in conversations with Johannes and his co-founder, and I understand their model for sustainability. So far I haven't found any non-free dependency. Now I don't recall about their app store but thought it was already or was going to be free, how's the status on that?
What @jonrichter says about a "central organisation" isn't really disqualifying it as free software. But I also would like a more decentralised set-up, which one could develop based on the CloudRon setup.
That said, with CommonsCloud.coop we decided to part ways and base our infrastructure on a combination of:
* an LDAP scheme prepared for decentralised nodes/clusters with users that can belong to collectives and have access to services at a node
* we built a webclient to manage the onboarding and allow for the decentralised management of collectives at a node
* ansible scripts, building on and contributing to the work of WebArchitects, for the quick deployment of NextCloud/Discourse/Phabricator + Zabbix monitoring + nightly backups + a synchronised local copy of the LDAP consumer
I'm sure our lead developer Chris Fanning can explain better; we're documenting on our Phabricator instance,. The code is on gitLab so far, but we will open a git repo at the Phabricator project platform soon.
I think an important difference lies in the sustainability model. The Cloudron team makes a living from maintenance fees and SaaS services. CommonsCloud is co-owned by its users through the cooperative and seeks to be sustainable based on contributions from users and collectives that want dedicated instances authenticating with their CommonsCloud account. Our vision is to emancipate users in this process, so they learn what is possible and decide+commit to make ongoing improvements and additional network services.

Thank you all for these wise remarks. Indeed it is tricky to find a suitable environment for managing cloud environments, if one doesn't want to stick to Kubernetes right from the start. Portainer was another alternative, but also has caveats at hands. By strictly binding to Docker Swarm, or single docker nodes, it gives us the ominous vendor lock-in the Open Container Initiative tried to avoid. I'm not certainly sure if we get there, soon. The technical stacks are often prepared and growing in very specific environments. The modularity of swappable components will still need to agree on certain protocols and conventions to do things.

In my understanding the Cloudron App store is not free, and as such its whole computing environment. When running a libre cluster environment, we want to make sure all parts can be reproduced locally.

Please feel highly invited to share views on this with the emerging LibreHosters network:

• Site: https://libreho.st
• Chat: https://riot.allmende.io/#/room/#librehosters:matrix.allmende.io

Is it ok if we keep this thread on topic (Creating self/community-hosted replacement(s) for GitHub), and open a new thread to discuss the hosting options for organizations wanting to run their own "cloud" in full software freedom? I, for one, have often found myself very confused by all this talk of Cloudron and Kubernetes and OpenStack and Sandstorm and other similar systems, and it would be great to have a thread specifically for pooling our knowledge of these systems, how they compare to each other, and what they are useful for (or not).

Just an fyi for those interested: hack.aragon.one

This thread has been a bit stale - and for our part (ValueFlows) inertia set in and we never moved from github. And would prefer to go en masse, all agreeing on where. But, now a new reason: https://help.github.com/en/articles/github-and-trade-controls. Yipes! There are reports from people from Iran who actually got kicked off.

So I thought I would check here -- did anyone make the move? If so, where to?

You could consider our GitLab server at https://git.coop/

@chriscroome thanks a lot for the offer. We're on the run today but will have a lot of questions, starting tomorrow. Should we chat here, or elsewhere? @lynnfoster is also starting a github issue in the valueflows org, and we could alternatively chat there, or wherever else you like.

Anybody else here interested in some gory details about this topic?

https://github.com/valueflows/valueflows/issues/526

(We thought it was fitting to post a "get out of github" issue on github...)

Here is fine, or there is a thread on the open CoTech forum we could use:

https://community.coops.tech/t/git-hosting-for-co-operators-gitlab-at-git-coop/544

Or open a ticket by emailing support@webarchitects.coop

FWIW another host you could consider is Codeberg, a non-profit entity based in Germany, which is funded by donations (and maybe paid memberships?). Their platform is an instance of Gitea. When I asked on the fediverse, the Codeberg folks said they are working with the ForgeFed team: https://codeberg.org/

A similar, slightly older service is NotaBug, which is an instance of Gogs. Not sure whether either of these projects is working with ForgeFed, but it seems likely:
https://notabug.org/

From what @chriscroome has described, Git.coop seems like an ideal solution for teams like social.coop, with a fairly bounded membership. But perhaps Codeberg or NotaBug might be more suitable for an open-ended standards collaboration like ValueFlows?

Our rough plan is to move first to someplace easy, which means probly a gitlab instance where we can import at least most of the current contents.
After that, experiment with a federated setup. But to do that, we want to do a ValueFlows treatment of all of the work and components that go into it, including code commits, repos, issues, PRs, etc etc. In other words, full dogfood.

@chriscroome if you look at that ValueFlows issue, the early returns on moving to git.coop are favorable. And I've looked around a bit and like what I see.

But one early question, which you can see in that thread, is about open registration for contributors and commenters.

I see on git.coop,

If you join as an organisation we will whitelist your domain so all your members can register accounts, if you join as an individual we will create a @webarch.coop email alias for you to use to register an account here.

We would join as an organization, altho we are not a formal one, and nobody has valueflo.ws email addresses. Everybody has some other home base and email address. But because of its domination of git hosting, everybody has a github account.

How would this work for us at git.coop? Can everybody freely register at git.coop and participate in our repos? Contribute code? Raise and comment on issues? Review pull requests? etc etc?

Would you need to give them webarch.coop email aliases? Will that be a problem? What would each individual need to use that email alias for? Would they need to be aware of it at all, or is that just for your internal use?

I'm trying to explore this situation and so far, Lynn is having a hard time understanding what I am trying to ask, but maybe you can respond with enuf details that we can sync up better, and I can ask better questions?

Related question: the signup form at git.coop says you can sign in with github. If our contributors do that, do they have smoother sailing to our repos?

I've been trying to track this to some degree in a page on the P2PF wiki, which I linked in the OP: https://wiki.p2pfoundation.net/List_of_Community-Hosted_GitLab_Instances

Two projects that are also relevant here. One is working on the [ForgeFed standard](https://talk.feneas.org/c/forgefed), which is making good progress on using ActivityPub protocols to allow federation between code forges. Also worth looking at is the [Sourcehut code forge server](https://sourcehut.org/), which uses Git and email protocols to facilitate federation between instances of the Sourcehut software.

We're planning to move first to something like git.coop, but don''t have answers to our questions from @chriscroome yet.

After that move, we plan to experiment with ForgeFed with one project to begin with.

Sourcehut is also interesting but sr.ht is based on the US and is subject to US sanctions.

Hi Bob, 6 days ago you asked:

Should we chat here, or elsewhere?

On the same day I replied:

Here is fine, or there is a thread on the open CoTech forum we could use:

https://community.coops.tech/t/git-hosting-for-co-operators-gitlab-at-git-coop/544

Or open a ticket by emailing support@webarchitects.coop

Sorry if I have missed other questions, I find the threading on Loomio hard to follow.

@chriscroome yeah, the threading on Loomio can be goofy.

Here are my questions (copied from upthread):

if you look at that ValueFlows issue, the early returns on moving to git.coop are favorable. And I've looked around a bit and like what I see.

But one early question, which you can see in that thread, is about open registration for contributors and commenters.

I see on git.coop,

If you join as an organisation we will whitelist your domain so all your members can register accounts, if you join as an individual we will create a @webarch.coop email alias for you to use to register an account here.

We would join as an organization, altho we are not a formal one, and nobody has valueflo.ws email addresses. Everybody has some other home base and email address. But because of its domination of git hosting, everybody has a github account.

How would this work for us at git.coop? Can everybody freely register at git.coop and participate in our repos? Contribute code? Raise and comment on issues? Review pull requests? etc etc?

Would you need to give them webarch.coop email aliases? Will that be a problem? What would each individual need to use that email alias for? Would they need to be aware of it at all, or is that just for your internal use?

I'm trying to explore this situation and so far, Lynn is having a hard time understanding what I am trying to ask, but maybe you can respond with enuf details that we can sync up better, and I can ask better questions?

Related question: the signup form at git.coop says you can sign in with github. If our contributors do that, do they have smoother sailing to our repos?

If you would prefer that I move the questions to one of those other locations, let me know.

Hi Bob, my suggestion for you would be to do things the way social.coop is doing them, we could add the valueflo.ws domain to our mail server, you could then use our mail server to create @valueflo.ws email aliases for the members of your organisation and then people could use their @valueflo.ws email addresses to sign up for accounts at git.coop.

This way you would be in control of who has the ability to create valueflo.ws accounts.

The GitHub login was added to aid migration of projects from GitHub rather than to allow account sign on / creation from GitHub.

@chriscroome

then people could use their @valueflo.ws email addresses to sign up for accounts at git.coop.

Nobody has a @valueflo.ws email address. That's the problem. Valueflows is not a formal org, it is a (github for now) set of related projects whose only close-to-formal org identity is as a github org.

the threading on Loomio can be goofy.

I believe it's possible to turn comment threading off, and I strongly recommend doing so in a group like this. It has a tendency to make threads metastasize into unwieldy mega-threads instead of sticking to a topic, especially when folks are dialing in via email a lot and can't see the comment they're replying to in the context of the rest of the thread.

sr.ht is based on the US and is subject to US sanctions.

OK, but that doesn't stop you hosting a Sourcehut instance in whichever jurisdiction pleases you, and interacting with all the projects on sr.ht and other instances via Git and email (which seem to be sanction-proof so far).

Nobody has a @valueflo.ws email address.

That is why I suggested adding the valueflo.ws domain to our mail server so you could create aliases for people.

@strypey

that doesn't stop you hosting a Sourcehut instance in whichever jurisdiction pleases you

I spose that's possible, but I think way too big a leap for valueflows in one move from github. And I think a forgefed experiment would be a more interesting step for the future.