Autonomous Infrastructure

You should make this a proposal.

Would be amazing. This thread: https://www.loomio.org/d/Z7dPrGBI/business-management-software might signal* that it's a challenge to operate with such autonomy.

*like me, have others switched to Signal : https://en.wikipedia.org/wiki/Signal_(software) ?

Hi Steven. I've seen discussion amongst those that know about such stuff that Signal isn't all it's cracked up to be. Lots of talk about http://matrix.org being a better option...

Absolutely! Two threads to this:
Physical infrastructure - DCs, racks etc - Brighton DX one co-op example
Cloud infrastructure - thread already on Slack about co-op cloud

Just to say that when we were setting up earlier this year (We Are Open Co-op) our aim was to be entirely self-sufficient and away from proprietary providers. We attempted to self-install Etherpad, and to use the Sandstorm.io range of apps.

Unfortunately, practicalities and pragmatism won the day, but we'd definitely be up for collaborating on this. We've got a face-to-face planning session from today (Weds) until Friday so I'll make sure we discuss this at some point.

Do we have a sense of who would take the lead on this? I think an audit of what people are already using might be a sensible starting point :)

I think Doug's suggestion is good to I have created a wiki page where co-ops can list what applications they are self-hosting for their internal use, I think I have added the main ones Webarchitects use but have probably forgotten several, I also added a couple that I think Outlandish and Agile are using.

In terms of Brian's point:

I worry this is purity for purity's sake, and that actually we could get more done if we allow ourselves to get a little dirty.

Are you saying you are against us having an aim to "to self-host our own internet infrastructure"?

I'm not proposing we should host everything ourselves, just that we should aim to, self-hosting Loomio doesn't appear to be a practical option at this time, for example.

I suspect that if we all document what systems we are all already using we might find that between us are already self-hosting everything we might need...

Stepping back a bit, would it help to start a wiki page for ideas regarding what our overall aims might be? I missed the session at Wortley Hall on this but think it might be good to have some concrete things on the agenda for the next time we all meet face to face?

Thanks Chris! I actually meant what people were using overall as we don't know what we don't know. For example, it's fine to make the decision to use Trello once you know that Jira and Wekan exist, but if you think that's the only app in the world that works like that, it's not an informed choice.

I like the fact that the wiki page is set up in a way that not only lists the app but also what it does does. I've hacked the format to add a section on the apps that We Are Open Co-op would like to transition towards; many of them are a click away if we had a collective instance of Sandstorm.io :)

---

Aside: our co-op is a consortium of limited companies, so we have a 'bring your own infrastructure' for the most part. My company, Dynamic Skillset, basically runs off Google G-Suite as it makes my life easy. If anyone has experience with other solutions (e.g. MyKolab) please message me off-list!

Thanks @dougbelshaw for the wiki edits, I have made it broader still, hows this?

This is a page to document what self-hosted systems co-ops are using, would like to use, non-self hosted systems they would like to find alternatives for and systems they are hosting for others.

Also there was a session on this at Wortley Hall and I guess some things from that page could be merged in?

Just to remain as the sole dissident voice in this conversation :)...

I'm not against hosting our own tech, per se, I'm against expending effort for bad reasons.

Doing it to bring business to CoTech members feels like a good reason to me. But doing it to gain some kind of digital autonomy just doesn't do anything for me. You all know much more about it than me, so I'm out of my depth here, but my understanding of cloud services is that their USP is the effort and cost they save you compared to self hosting. If another coop could host all Outlandish's tech, then great, but if we need to maintain it ourselves, less great.

Maybe if you could just clarify that point for me (in terms of what would this mean for Outlandish - what extra effort we would have to put in to running our own systems) then perhaps I could drop out of the chat and leave you all to it :)

There are really two threads to this conversation:

1. has 'infrastructure' meaning tools and services for internal use that CoTech members currently buy/use from cloud/SaaS providers, and (possibly) some of the infrastructure supporting these further down the stack.

2. has 'infrastructure' meaning the infrastructure that supports services that CoTech members currently use to provide service to customers and ,may bundle in their products - eg servers for web hosting.

While autonomy, independence and capturing more of the value chain are relevant to both:

1. • being about internal matters - has the discussion focused on autonomy

2. is more about value chain.

2 is also where my attention personally is focused. I'm not too bothered about 1 personally - although only because it doesn't affect me much.

So in answer to @brianspurling "why bother":

Two business reasons:

a) We coops can add more value if we run more of the stuff. Simple as that.

b) In many cases AWS and similar are expensive and not actually that easy to use.

(There is also an argument that says that Amazon is a ghastly company, but you don't have to go there. There are - in some cases, not all - plain old business reasons to use alternatives.)

Now, a coop cloud provider could, if done well:

Provide good vfm
Provide great service
Also, as a side point be coop and green(ish)

It would also be cool.

Shaun

While I understand the point, I think it could be argued that (a different sort of) autonomy and independence comes from NOT having to maintain your own infrastructure and applications.

Because I care very little about the general privacy issues that seem to be driving this conversation, I for one would prefer us to take the route with minimal effort involved, so we can focus our efforts instead on things that (IMO) really matter. But I understand that others will feel differently :)

I worry this is purity for purity's sake, and that actually we could get more done if we allow ourselves to get a little dirty.

It's not just about autonomy/privacy. It's crucially about value chain. An important route for growth for some (not all) digital businesses is to capture more of the value chain. That's the thinking behind the DX model. We do not have to be confined to the top (application) layer of the stack. We don't have to be just consumers of bandwidth and hosting, we can play a role in providing them and co-operativise (sorry) more of the layers.

Coops too often are confined to selling other people's stuff (or selling their labour to work on other people's stuff). (Eg The Phone Coop just resells access to other people's networks.)

Yes, definitely get that. Sorry, I thought we were talking about hosting the tools we use just for ourselves... ?

I'd like to express appreciation for this conversation, on both sides. Yes, other things being equal, great to be autonomous and independent on the software as well as other fronts. But with @brianspurling we know that often things are not equal, and that self hosting (a bit like food self-sufficiency back in the 1970s) can drain all one's time rather pointlessly.

I wonder if a great aim -- I think hinted at by some of the contributions -- would be to grow whole ICT systems that are cooperative in execution as well as in spirit. If we want to supplant -- or at least provide an alternative to -- the major capital-driven businesses, we will need to be very efficient ourselves, and use as many economies of scale, and economies of scope as we can.

We have to tackle, as we are addressing elsewhere, business models for all cooperating players. Where software is open source and self-hosted, how are the creators recompensed? Various possible answers, but I guess that should be on a different thread. Or is it, already?

While I have a limited understanding of what self-hosting would take at a technical level, I do find this - https://www.youtube.com/watch?v=112d45b03hM (and apologies in advance to the more politically sophisticated among us... ) - as good a reason as any for why, in a cooperative context, it's a 'good thing'

Dmytri Kleiner has a principled position, which is fine in itself. A bit like Richard Stallman ;) My (possibly naive) understanding is that Dmytri would advocate building systems where there are no intermediate servers, meaning that information is not stored anywhere between the people communicating, and so in principle the information communicated cannot be exploited in any way. Which is a noble aim.

Personally, I see it as quite acceptable for people to share at least some of their communication systems with other people. If you use a number of devices, it can be hard (though perhaps not impossible) to keep them "in sync" without the help of a server in between. If those servers are run by a worker coop -- as in platform coops -- and directly answerable to each user rather like a member coop, what's not to like? Perhaps, mainly, as I was saying above, what can be improved are the economies of scale and scope.

Following @dougbelshaw if it's too hard to do, or simply uneconomic, people will use the commercial tools so as not to disadvantage their own work. So the challenge, as I see it, of autonomous infrastructure is just to get it working in a way that has minimal costs both in terms of money and time for users. It needs to be as easy to learn and use as commercial alternatives. That's where some current open source software has a weak spot.

Can't say more than ditto to all of the above.... and that I'm a sucker for principle :)

I am in favour of autonomous infrastructure as an ultimate goal. I agree fully with the ideas expressed by the likes of Dmytri Kleiner as to the overall structure of the internet. I talked to @chriscroome a little about leveraging, say, OpenStack to provide a cloud offering hosting on hardware owned by cooperatives.

However, I feel it should be a strategic goal not a tactical priority. Its the kind of thing you chip away and have to plan with clarity if you want to get anything done without it hampering other more immediate goals like simple operational effectiveness.

I'd want to move this discuss therefore to more pragmatic territory. Is there one type of thing that CoTech could do, and do well, to provide autonomous infrastructure for cooperatives? This one thing should be then done and if found to be worthwhile expanded into other areas. It could be, for example, email, or chat or file sharing. It would have to do this roughly as well as the big players in this space and there are some clones which are decent. Then, with this success under the belt, we can begin to talk about transitioning another system. This is how you would plan an IT project and I see no reason why this shouldn't be done here.

A working system operating at scale is a far more important thing than to merely ask if such a thing is desirable or plausible.

I'd want to move this discuss therefore to more pragmatic territory. Is there one type of thing that CoTech could do, and do well, to provide autonomous infrastructure for cooperatives?

Yes, I think we should have a shared instance of Sandstorm

A working system operating at scale is a far more important thing than to merely ask if such a thing is desirable or plausible.

+1,000,000. I'd be fine with just starting with Etherpad, as there's not even a commercial offering that does such a good job for getting stuff done in a decentralised, realtime way.

... to add to @shaunfensom points:

An 'owned' infrastructure would/could be a distinct + vis a vis PR to both potential members/collaborators and clients.

Not sure how relevant this is, but I've just come across what looks like a great collection of Libre software, with what looks like a donation model (if you read French)... https://framasoft.org/ -- does anyone know these people?

Having infrastructure/hosting/sysadmin/devops provided from within CoTech sounds very attractive. While AWS has a million services (many of which are pretty cool and innovative) it's pretty bewildering unless you make it your life's work to understand it.

Outlandish has traditionally seen this area as a but of a pain rather than an opportunity - basically because none of us have ever been that passionate about it. We've been very pleased to work with @chriscroome on a few projects since the meet up and he's got us much further than we'd have got on our own for the same time/money.

It's a shame that Blaze Hosts seem to have gone of the radar. There were (apparently) six of them running a hosting co-op in Manchester. Maybe they can be reached somehow.

It would seem that with Web Architects, Broadband Co-op, (maybe) Blaze Hosts and whatever capacity the other members have we have some good expertise in this area.

Would anyone be up for leading some research into a potential business model for pooling services in this area? I'd be up for contributing but not sure i really have the skills or capacity to lead it.

We've also got something in the region of £200k EU funding to investigate how to make Docker work well and would be keen on collaborating with others.

@harryrobbins said:

Having infrastructure/hosting/sysadmin/devops provided from within CoTech sounds very attractive.

I agree and had a good chat with Alex about this at Wortley Hall and we have a Coop Cloud wiki page and #coopcloud slack channel where we are discussing this.

However this wasn't why I started this thread and it's now clear that there has been some misunderstanding -- all I was intending to suggest with this thread was that we should have an aim to run our own infrastructure for our own internal use as a network of Co-operative Technologists, and to a large degree we are already, our wiki is self hosted, so is our email list and so will be our web site when it is launched and so is the dev site at the moment.

If it is practical and easy to self host other things in the future and if there is a demand for them then let's do it, for example Etherpad has been mentioned a few times and perhaps having a pad.coops.tech site would be nice someday, but I recall hearing in IRC at the time that Riseup set up their pad that is was quite a pain, but that was years ago. Hosting our own Loomio instance and perhaps even Rocket.Chat or something might also be good sometime down the line but I'm not suggesting we should do these things now.

Setting up a MediaWiki site for the group during one of the sessions, while hung over and half asleep, was quick and easy for me because I have been doing it for years, in contrast at the other end of the spectrum, producing an alternative to YouTube and Vimeo to host our videos would be far from quick and easy to do, so it would have to be something for a very long term wishlist, if we thought it was a good idea -- it simply isn't practical and as has been discussed on other threads I'm also keen to spend more time getting stuff done that talking about getting stuff done 😃

PS Exploring Docker sounds interesting, but it isn't a technology I have found time to play with yet.

@harryrobbins said

Outlandish has traditionally seen this area as a but of a pain rather than an opportunity -basically because none of us have ever been that passionate about it.

Lucky Outlandish now has someone in the mix who really is passionate about it!

I've been researching Open Stack quite a bit over the last week and it seems that, given hardware and fully understanding it, creating a AWS compatible cloud which includes the three MVP level features me and @chriscroome talked about at Wortley Hall (virtual servers, databases, storage compatible with S3) should actually work "out of the box". Plus Open Stack already has mature plugins for the most popular platforms for infrastructure as code: Ansible, Salt and Terraform. It wouldn't take more than a couple of lines change to adopt, say, some of Outlandish's Ansible provisioning scripts to use this https://docs.ansible.com/ansible/os_server_module.html

I think me and @chriscroome will kick the tires on this in the new year when he comes in to do some stuff around provisioning at Outlandish.

Just to chime in saying that I think this network coming together to offer a co-operative & open source saas/ cloud offering would imho be absolutley brilliant (and I'd be a customer, as would I imagine quite a large niche of companies/ NGOs etc who would much prefer to not routinely give Dropbox, Google etc money and would much prefer to routinely give money to nice co-op people maintaining open source stuff).

And also to add that Sandstorm rocks and everyone should check it out (and this network should have an instance of it).

@chriscroome @alexwa check this out https://ioo.coop/2017/01/14/prospectus-ioocloud-a-cloud-services-cooperative/ (although I see you've already seen it Chris).

Thanks, that proposal is very much in line with the Sheffield based https://darkpeak.org/ co-op (I did try to get some of them to come to Wortley Hall) but I guess more ambitious (and US based), I think it is a great idea and I'd love to see a lot of co-ops of this nature.

What we have been discussion in terms of a co-op cloud was more like a co-operative version of Amazon Web Services -- aimed more at other co-ops rather then individuals, but of course there would probably be some overlap with these things.

As I also commented over on ioo see also IndieHosters - they aren't really a co-op (at least not yet, think it's basically 2 people), but offer similar "hosted open source apps" stuff (they host the Discourse we've got at https://discuss.open.coop/ )

Over on the Commons Transition Loomio the topic of http://www.beyondbroadband.coop/ has come up, under the topic of "Commons trasition: what about the IT infrastructure?" [sic]

Is there a way of coordinating the discussion? Or is all we can do to have people in both?

@asimong I was involved in the beyondbroadband.coop work and know the INCA people well (INCA was originally set up by CBN). I don't know how to co-ordinate the discussion. Are there specific questions being asked?

The thread was Started by Michele Kipiel 6 days ago, and introduced as follows:

"there's a great deal of debate in the P2P community about the ways the commons transition will take shape in fields as diverse as energy production, farming, housing, industrial production and so on. Almost all proposed solutions take the IT ifrastructure as a given: the internet is assumed to always be there for us to use, no matter what. This seems to me like a naive and dangerous assumption, as it greatly downplays the importance of ISPs (which are private companies) as both gateways and gatekeepers of our access to the internet. There are many ways both governments and private institutions could get in the way of the commons transition by disrupting our communication channels.

Has this been debated here before? Are there any resources/articles/papers around this problem I should be aware of?"

Graham -- @graham2 -- was the one who brought up beyondbroadband.coop

Are you here, Graham?

Michele is right to raise it and that it shouldn't be taken for granted. But there's so much to unpack there. ISPs don't control the infrastructure, they merely use it. Fairly easy (well, relatively) to construct a coop ISP (The Phone Coop, GreenNet) but getting control of the infrastructure is a much bigger deal. Good example is b4rn. Then once you have the infrastructure you need to deal with the Internet itself - peering vs transit and the role of Tier 1 ISPs like Level3. A huge area to consider. What should our role be in this debate?

I think alternative infrastructure is a fascinating topic but I how we don't need to build our own internet anytime soon.

Personally I see less capital-intensive (more labour intensive) businesses (such as agencies) being more obvious opportunities for cooperativisation in the medium term.

There seems to be a fixation in the platform co-op movement at the moment with targeting industries that require $1bn investment to get started.

We don't need to build our own Internet because it is already made up of bits. It's ideal for adding bits, which is how come b4rn has been successful. LINX is a mutual - indeed a co-op. Many other IXs are mutuals. Adding coop bits to this infrastructure is entirely feasible. Restricting coops to the top layers of the value chain leaves us vulnerable, limits the transformation we make. We end up like The Co-op, and The Phone Coop, selling other people's stuff.

I'm all in favour of co-ops owning and running as much of the Internet as possible but @harryrobbinss have a valid point -- when Uber fares only cover 40% of the cost of a ride and venture capitalists are prepared to spend $2 billion a year to give people cheaper fares and wipe out the competition it's rather hard to see how a platform co-op or even government funded public transport infrastructure can afford compete...

I have asked to join the Commons Transition Loomio group.

Agree that industries requiring massive investment are much harder. But it is a myth that digital infrastructure is all about giant datacentres and massive network spend. Two important points: a lot of hosting inevitably will move to the network edge where it will need shared, neutral space (opportunity for coop model); peering will similarly move towards the network edge (for similar reasons), opening opportunity for local access networks to be viable at smaller scale.

Possibly being a bit slow here but only just noticed http://cocloud.coop @CoCloadcoop

Thanks @shaunfensom -- that's new to me, here are some working links:

• https://opencollective.com/cocloud
• https://twitter.com/CoCloudcoop
• https://www.loomio.org/g/yhLxDvrF/buy-twitter-cocloud-project
• https://ioo.coop/2017/01/14/prospectus-ioocloud-a-cloud-services-cooperative/

But these are not very enlightening -- I guess it is all happening in the closed Loomio group, I have asked to join it.

Gitter looks like a really interesting alternative to Slack, it has just been brought by GitLab and it due to be Free software by June 😃