problems with upgrading old installation

So to start with the very basic beginning:

cd loomio-deploy/

docker-compose pull

 docker-compose down

 docker-compose run app rake db:migrate

[...]

Migrating to UpdateStanceDataForMeetingPolls (20190827234738)

== 20190827234738 UpdateStanceDataForMeetingPolls: migrating ==================

rake aborted!

StandardError: An error has occurred, this and all later migrations canceled:

The single-table inheritance mechanism failed to locate the subclass: 'FormalGroup'. This error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this column if you didn't intend it to be used for storing the inheritance class or overwrite Group.inheritance_column to use another column for that information.

(I wasn't able to paste the whole stack, led to internal 400 and 422 HTTP errors on the Javascript console, but no message visible to the user)

Hi @Holger Wessels. Sorry I've not had any time to look into this until now.

The key part of the log is: The single-table inheritance mechanism failed to locate the subclass: 'FormalGroup'.

The easiest way to fix this might be to bring the classes back, empty.

I've just commited this solution to the master branch, and a new latest image will be built in the next hour, and you should be able to switch from stable to the latest docker image in docker-compose.yml and hopefully that will help you through.

I would be bracing myself for maybe one or two more problems like this given the age of your database.. but who knows, maybe this will be the only issue 🤞

Well, I'm getting further, but not until the end yet. :-) With the latest docker image I'm the migration stops with a different type of error, this time a missing function of which I don't know how important it is:

Migrating to UpdateStanceDataForMeetingPolls (20190827234738)

== 20190827234738 UpdateStanceDataForMeetingPolls: migrating ==================

rake aborted!

StandardError: An error has occurred, this and all later migrations canceled:

undefined local variable or method `hideresultsuntil_closed' for #<Poll:0x000056238e27a1e8>

Could it be a better strategy to try to update in smaller steps to versions in between (like in 3 month steps), if that is possible?

Well, to answer that to myself: it appears that the versions between March 2019 and v2.1.10 are not on docker hub, so not so much luck to pretend I did frequent updates 🙂

Hi @Rob Guthrie , any leads on hide_results_until_closed?

yea, it's taken us too long to move to proper versioning, which would have really helped in cases like this. Oh well, it's here now.

I'm pushing a fix to master (docker:latest) now. Should be up soon. Thanks for finding these migration issues for us.

Well, slowly moving forward. But here's another one:

Migrating to UpdateStanceDataForMeetingPolls (20190827234738)

== 20190827234738 UpdateStanceDataForMeetingPolls: migrating ==================

rake aborted!

StandardError: An error has occurred, this and all later migrations canceled:

PG::UndefinedColumn: ERROR: column stances.revoked_at does not exist

LINE 1: ...s"."poll_id" = $1 AND "stances"."latest" = $2 AND "stances"....

^

: SELECT "stances".* FROM "stances" WHERE "stances"."pollid" = $1 AND "stances"."latest" = $2 AND "stances"."revokedat" IS NULL ORDER BY "stances"."created_at" ASC LIMIT $3

Whoops, posted early.

You can just comment out line 3 in db/migrate/20190827234738_update_stance_data_for_meeting_polls.rb and run the migrations.

I'm not sure how to fix this/make it go away for for others yet. Scheduling it as a background job when the app boots would do.
But.. maybe I'll see if someone else wants to do that for me.

Hi Rob, that actually brought me through the database migration :-)
Now something appears to be running. But I'm getting time out errors and 500 errors, and

docker-compose logs -f

shows a repeating pattern like this:

loomio-worker | 2020-07-15T12:09:10.361Z pid=7 tid=grwby67hr WARN: Redis::CannotConnectError: Error connecting to Redis on localhost:6379 (Errno::EADDRNOTAVAIL)

l

Nice. We'll get there.

Looks like your config (env or docker-compose.yml) is missing some new variables... particularly the REDIS_URL. Try updating your docker-compose.yml to match what is the current master on loomio/loomio-deploy

I can already feel it, the end is near ... 🙂 But: I updated loomio-deploy (also .env to postgres 12.2, and both TAG=latest), and ended up with an error similar to another recent thread (but in this case I'm already up-to-date):

loomio-worker | 2020-07-16T00:18:07.801Z pid=7 tid=gqfkyrvc3 WARN: PG::ConnectionBad: could not translate host name "db" to address: Name or service not known

(leads to continuous restarts and 500 errors displayed).

Does your docker-compose start with a version like master?

Hm, I‘m not sure what you are referring to with master. The loomio-deploy ist from the Master branch, and I’ve just updated it (to avoid more version difficulties). Last update must have been a month ago or so. the docker-compose command are just the normal ones, without any version. From my feeling everything should be as new (latest) as possible ...

Sorry, was on my phone, trying to answer quickly. Usually db not valid hostname is a problem with the docker-compose.yml file being in the older format. The new format starts with version: '3' on it's own line, then a services: section, with db: and app: and such indented under that. Do you have that?

As seen here: https://github.com/loomio/loomio-deploy/blob/master/docker-compose.yml

Hi Rob, yes I have the exact same version from the master branch. Any specific parts of the log file or configuration files which could explain this? Shall I add db to the hosts file on the server, or is that a docker-internal reference? The postgres version has changed from 9.6 to 12.2, maybe some additional configuration steps are necessary due to that?

And, when I now run update.sh, also the migration step fails in very beginning because of the problems with db hostname
[...]

rake aborted!

PG::ConnectionBad: could not connect to server: Connection refused

Is the server running on host "db" (172.18.0.2) and accepting

TCP/IP connections on port 5432?

I would expect to see some postgres errors.. and actually.. I wonder if there is some you're not seeing. Also.. try modifying your .env file so that you're using postgres image 9.6 rather than.. whatever the latest is... because your database will be 9.6 I expect.

Oh, sorry, I expected an automatic migration here, too ... Soon I'll probably get along way better with docker containers and their construction... So, now the db host problem seems to be gone, but I'm still at 500 error. Look like I broke something during updates...

loomio-app | Started GET "/dashboard" for 79.224.162.67 at 2020-07-16 10:25:11 +0000

loomio-app | Processing by ApplicationController#index as HTML

loomio-app | Completed 500 Internal Server Error in 6045ms

loomio-app | Errno::ENOENT (No such file or directory @ rb_sysopen - /loomio/public/client/vue/index.html):

No such file or directory @ rb_sysopen - /loomio/public/client/vue/index.html means the vue client did not build or could not be found.

Are you running stable or latest?

Actually, I posted that last comment too early.

Honestly, I don't get this error. The vue client is built during the Dockerfile build stage, so before the image gets to you. (assuming you're running :latest).

It's very strange to me that it's not there.

Side note: I just found a problem where app should have had "depends_on: redis" added, so I've added that to master now.

I'm running latest, now with your last addition. The funny thing is that the file is there somewhere and readable (/var/snap/docker/common/var-lib-docker/aufs/diff/e5de098ed06bdabead2ada2aaf96e78166a8c5c39be7950154a38e9cdb176683/loomio/public/client/vue/index.html), but somehow the relative path is not working. I tried also en-us locale in the browser, but no change. The machine has a very tight memory and cpu setting, it was just enough for the old version a year ago, but I sometimes get requestTimeoutExceptions (>15sec) usually in sprockets. Do you think those problems could be related? the 500 usually comes quite fast ...

oh dear, I think this line needs to be taken out of docker-compose.yml:

- ./client:/loomio/public/client

I've updated loomio-deploy master just now, so you can git pull rather than editing if you like

(sorry, that was before your last comments) Still strange. I've upgraded the server to 2 vCpus and 2 GB RAM, but the problem remains.

Yes, that was the break-through! :-) Now there's a beautiful new looking loomio (and a slightly bigger server) ! 👏Thanks for all your effort!

Thank you, really, for persisting. We found a lot of issues together... and I can't do it alone.

If you wanted to go back and edit your comments so only the relevant lines of stack trace remained, and maybe put them into code fences, that would form a pretty good guide for anyone else who later decides to upgrade an old Loomio install.

That's a good idea, I'll do that tomorrow morning 😴

Done. Was also a good exercise in reading logfiles 😉