Struggling with Docker, is there another way?

P.S. I worked out how to run it on a different port - I turned off apache, which was competing for port 80 of course, and ran docker-compose up which caused the SSL certificate to be generated (for which it must be running on port 80). Then I went into docker-compose.yml and "80:80" to "81:80" and "443:443" to "444:443" , did "docker-compose down" and then "up -d" again and lo and behold I could access loomio on port 444 and (after restarting apache) could access my other sites on 443! Happy days. I'm having an issue with email now though, which I'll start a new thread for. I attempting to debug now with the help of a more knowledgeable person, and will post any info that might help others in that thread.

@robertguthrie if I get through this I would like to compile what I've learnt into a troubleshooting guide - would you be interested in that being added to your repo?

Hi @naomir , totally appreciate your post, thanks for writing it.

I'll just add my reasoning first, then address your questions.

I've chosen the Docker path because it gives you a secure, full featured loomio install in the least number of steps. That is: You get SSL/HTTPS, inbound email, simple DNS settings, background jobs and crontab, redis and postgresql, and even a simple backup system.

Our crowdfunding campaign in 2014? promised a "one-click" install process - which is actually kind of impossible, but this is the closest I've been able to come to providing a simple to setup processs for hosting your own Loomio install.

Personally I find that docker keeps the level of complexity down.

If you're trying to run this alongside other services, or in a custom way, then it will get much less straightforward - that's just how it is. If you don't use Docker, then you lose the ability to easily update your system if I change a requirement. (like when I added redis to the stack 4 months ago, or when we moved from FAYE to actioncable last year).

I try to spend only a small amount of my time on the devops side of Loomio, becuase I want to spend as much time as possible writing good features and UX.

I think that running your own instance is always going to be something that requires a high level of server/linux/database experience, if you want to do it really well.. I've done my best to make it easy to get started, and also keep loomio.org running so people don't have to do this at all.

That said, I really like it when people such as yourself come along and actually use this work I've done. Sometimes I wonder if anyone is using it - because most people don't communicate with me... and we very rarely get any contributions, which I understand - because it is hard to contribute to a complex project.

I'd be happy to add/contribute to documentation about running your own production server. Have you seen the guide on setting up your own development environment? That covers quite a lot of this. (and may also need updating)

The docker guide does not work for a new blank server, because the command docker exec -ti loomio-db su - postgres -c 'createdb loomio_production' fails with Error: No such container: loomio-db
This is obvious, because no container is running at that time. I tried with the docker-compose up, but it leads to a complete mess I think. At the end I only get "500 Server Error" with no useful log information.

Ok, I'll run through a setup from blank again and update the guide. If I may ask, where do you plan to use Loomio?

I am in the same situation. I appreciate you taking the time out of your day to do this :) Personally, I plan to use Loomio for a project I am helping organize development for named Music Stash.

We would like to use it for the austrian burningman community. :heart:

Ok @klauskobald and @brendanbutler please try this:

docker-compose up -d db
docker exec -ti loomio-db su - postgres -c 'createdb loomio_production'
docker-compose run app rake db:setup


@robertguthrie
I've tested with these new commands in replace of database integration; it worked flawlessly up until the docker-compose up -d after the Cron tab is edited. No errors bringing it up, until the very last container (let's encrypt):

And I get 502 bad gateway errors when browsing to the domain

It can take upto 5 minutes for the app to boot. You will see bad gateway until then. I'll add that to the readme. You can see it boot with the logs command.

Alright thanks so much, I'll update you in a few minutes then :)Do I need to worry about that letsencrypt error?

Also you need to be sure that port 80 is open and your dns is correct for letsencrypt.

Boarding a plane. I'll check back later today

Port 80 and 443 are both open; using a Google VPS and I had selected both to be open when creating it. DNS is also set up (other than MX, I was going to set that up when I had it all working; is that maybe the problem?)

Safe travels!!!

Alright, still getting the 503 error and checked the logs and saw a bunch of various errors. Most are regarding the let's encrypt, but I saw some that weren't related. I am on the way home now, once I get home I will try copy pasting the entire error log into a Pastebin

Got home. The logs are VERY large,
I am unsure what in here is relating to my problem; though I am assuming its not all those letsencrypt errors as that would only deal with the SSL (I should still be able to connect to http, where Chrome would mention its insecure).

The weird part I find is that LetsEncrypt did seem to work; the page has the secure lock and it says valid from today on:

YES!
And it helps watching the log, because it takes some while for building!
So I´m getting the login page and will continue tomorrow! Greetings from Austria! :clap:

restarting with docker-compose down; docker-compose up -d can sort out let's encrypt issues sometimes. Also docker-compose logs letsencrypt will show just logs from that container.

Restarting the docker-compose seemed to fix the letsencrypt; and it all got up with no errors. I have been watching the logs and I am seeing no errors, but it's been roughly 2 hours and I am getting 503 when trying to connect.

loomio-worker  | [Worker(host:4c6775a2dec8 pid:7)] 6 jobs processed at 0.0526 j/s, 0 failed
loomio-worker  | 2019-05-23T01:11:32+0000: [Worker(host:4c6775a2dec8 pid:7)] 6 jobs processed at 0.0526 j/s, 0 failed
loomio-nginx   | nginx.1    | 2019/05/23 01:28:50 [error] 84#84: *20 connect() failed (111: Connection refused) while connecting to upstream, client: 72.137.176.178, server: test......xyz, request: "GET / HTTP/2.0", upstream: "http://172.17.0.5
:3000/", host: "test......xyz"
loomio-nginx   | nginx.1    | test.....xyz 72.137.176.178 - - [23/May/2019:01:28:50 +0000] "GET / HTTP/2.0" 502 575 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.157 Safari/537.36"
loomio-nginx   | nginx.1    | test......xyz 72.137.176.178 - - [23/May/2019:01:28:50 +0000] "GET /favicon.ico HTTP/2.0" 502 575 "https://test.....xyz/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
74.0.3729.157 Safari/537.36"
loomio-nginx   | nginx.1    | 2019/05/23 01:28:50 [warn] 84#84: *20 upstream server temporarily disabled while connecting to upstream, client: 72.137.176.178, server: test....xyz, request: "GET / HTTP/2.0", upstream: "http://172.17.0.5:3000/"
, host: "test......xyz"

It would be a good idea to run docker-compose logs app to see just the rails logs - that's likely where the issue is.

After waking up and checking the website (I left the serverto do it's thing) I found this:https://i.godhub.xyz/05-23-2019-rLNLLRwiv1.png

Maybe this has to do with my cpu, it was near its limit when I started to install this. Maybe when I get my new server into my datacenter in the up coming days I can try again on a more powerful machine.

Interesting.
CPU power is unlikely to be the issue - RAM possibly. the CSS compilation is a first request only issue too, so after it's compiled it will be fine.

You can set the ENV variable RACK_TIMEOUT_SERVICE_TIMEOUT=30 or more to give the server more time to do work before killing the request.

Yesterday we merged the new client into the master branch, and over the next couple of weeks we'll fine tune it so it's ready for a public beta then replacing the current client. One of the many advantages of the new client are that it can be compiled at docker image build time (before you docker-compose pull the image), so startup time should be only a few seconds.

These days I never run any server software without docker. It's the most important server skill I have.

Thanks for taking the time to work on Loomio and tell me about it, I really do want to support you however I can - which is often a little slowly... sorry.

Thanks @robertguthrie, I appreciate you taking the time to read my post and give it such a considered answer. I think in a way it comes down to different mindsets. Like you say, it keeps the level of complexity down, but only if everything goes to plan - and then if it doesn't, it suddenly becomes very complicated, and the usual Linux knowledge isn't enough to fix it. I find it's quite unusual for things to "just work" for me - I often end up delving into them for one reason or another - so the inability to (easily) do that is stressful. It's the same way I feel if I ever go near an IPad - I keep hearing how easy they are and how they just work, but they make no sense to me, I'm like "just give me a shell and no-one gets hurt". I guess it's a sense of not being in control.

However, I am already gaining knowledge of Docker from this. which is indeed useful and becoming more and more necessary as it gains popularity.

I will have a look at the development site docs you mention, and may have stuff to add, although it might be another few days before I get a chance to work on it

I would recommend that you try putting the shared server requirement aside, and try setting up loomio on a blank VPS as described, it should only take 30 minutes to an hour.

I totally respect the approach of taking a different path though, that's how I learned everything I know - including the value of just sticking to the path :)

Another really kind and enthusiastic person is wanting to create and manage a set of Loomio debian packages. Is that something that you think would be helpful to you? https://github.com/loomio/loomio/issues/5283