Loomio

Issues with deploying to Heroku

Elias Westerberg PublicSeen by 20

I've installed Loomio on a virtual machine and managed to run it locally. However, after following the instructions on the wiki on how to deploy Loomio to Heroku I've come to a dead end with the application replying with ERR_CONTENT_DECODING_FAILED when I try to access the application at https://evolutetest.herokuapp.com.

I can however access the static resource /privacy.

heroku run rake db:migrate and db:seed seems to be working (some deprecation warnings) and the following message:

You are missing the following optional environment variables: ["AWS_ATTACHMENTS_BUCKET", "BING_TRANSLATION_APPID", "BING_TRANSLATION_SECRET", "NAVBAR_LOGO_PATH", "NAVBAR_CONTRIBUTE", "TAG_MANAGER_ID"]

They have been set to defaults automatically."

Any ideas?

Elias Westerberg

Attaching my Heroku config variables and running processes.

Elias Westerberg

I'm also adding a cat from the production.log

Elias Westerberg

Seems like the assets are broken somehow. The .gz-files generated during precompile are corrupt.

I've tried experimenting by removing heroku-deflater from the Gemfile.

Instead of content decoding failed I get: HTTP/1.1 500 Internal Server Error... and compiled assets (.js .css) also have the HTTP/1.1 500 Internal Server Error as the first line.

mix irving

I've recently seen the same errors in your log when trying to install on DigitalOcean, though not sure if they are fatal issues.

Did you have a read of the heroku-deflator gem docs. I'm not sure whether that gem is needed, but you'd probably have to remove config for it, honestly not sure if the gem is needed any more, i thought heroku managed compression themselves.

I'm not an expert here, but you could look at playing with settings in production.rb
e.g.

config.assets.compress = true

there might be something to look at with configuration of config.cache_store = :dalli_store
not sure

@robertguthrie can you advise better ?

Robert Guthrie

I've not seen this error before..

Thanks for uploading your production.log that seems helpful... There are a lot of scss compilation errors. I really don't know what's going on there though.

mix irving

@eliaswesterberg can you post what you see when you push to heroku ?

we see this :

       Using client_side_validations-simple_form (2.1.0)
       Using librato-rails (0.10.2)
       Using inherited_resources (1.4.1)
       Using activeadmin (0.6.1) from git://github.com/gregbell/active_admin.git (at 40c18cf)
       Your bundle is complete!
       Gems in the groups development and test were not installed.
       It was installed into ./vendor/bundle
       Bundle completed (1.62s)
       Cleaning up the bundler cache.
-----> Writing config/database.yml to read from DATABASE_URL
-----> Preparing app for Rails asset pipeline
       Running: rake assets:precompile
       DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /tmp/build_8fae03c3-c67f-49ce-9c0f-008a967b823f/Rakefile:7)
       DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /tmp/build_8fae03c3-c67f-49ce-9c0f-008a967b823f/Rakefile:7)
       DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /tmp/build_8fae03c3-c67f-49ce-9c0f-008a967b823f/Rakefile:7)
       DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released. (called from <top (required)> at /tmp/build_8fae03c3-c67f-49ce-9c0f-008a967b823f/Rakefile:7)
       Asset precompilation completed (161.55s)
-----> WARNINGS:
       Injecting plugin 'rails_log_stdout'
       Injecting plugin 'rails3_serve_static_assets'
       Add 'rails_12factor' gem to your Gemfile to skip plugin injection
-----> Discovering process types
       Procfile declares types -> web, worker
       Default types for Ruby  -> console, rake

-----> Compressing... done, 64.4MB
-----> Launching... done, v310
-----> Deploy hooks scheduled, check output in your logs
       http://loomio-clone.herokuapp.com/ deployed to Heroku

To git@heroku.com:loomio-clone.git
 + c89bb07...2967060 amend-config -> master (forced update)

@robertguthrie see our production.log attached... we've got the same errors

Elias Westerberg

@mixirving the push to Heroku looks the same (attached).

mix irving

definitely not serving those assets.
here's a comparison of a server of mine with yours.

note yours has some Status code 206 (could be worth a look?)
and length 1

the difference in cache time is due to a recent change for crowdfunding. you might consider finding that number in production.rb and changing it down so that once your problem is fixed the cache doesn't get in the way.
or learn how to dump the cache.

is it worth looking to see if the actual asset is compiled somewhere (with length>1) to check that it's not some serving problem?

mix irving

@jonlemmon do you have any suggestions given this evidence what might be up with Elias' setup?

Elias Westerberg

Also tried to precompile assets locally and pushing them to Heroku. Same error.

Elias Westerberg

Hmm... I guess assets are compiled fine anyway. Locally I can open them and they look OK. When I try to access them on Heroku however it seems they cannot be read. Something is concatenated before the actual file. Check this weird looking application-xxx.js for instance.

Elias Westerberg

@mixirving any other suggestions?
I can see that both my loomio s3 buckets are empty.
Amazon recently changed their policy on access keys so I had to set it up a bit differently than in the guide. Perhaps this is causing my issues?

mix irving

I'm honestly not sure.
I think this could a case of us needing to go through and trying to replicate the setup for ourselves and seeing if we can replicate the problem. Apologies we've not been able to support you more with this @eliaswesterberg. At the moment a lot of us are needing to focus of finding some paid contracts, so can't afford to do this.

The change in Amazon setup is a bit frustrating, will have to do some reading/ learning on that front. If you have any notes to contribute to the guide they would be most appreciatred.
At the moment our priority is working on getting a Docker setup up and running, to streamline this process.

Elias Westerberg

No worries, I understand that you are fully occupied with other stuff. It makes sense to focus on creating an easier installation for everyone instead on focusing on individual issues.

When we have something that actually works, I'll be happy to contribute my notes, but for now maybe my notes will just get people in the same kind of trouble :P

Anyway, another developer in our team is setting up a loomio production server in parallel, let's see if he has more luck with getting it running.

I'll report back when we have any progress.

Elias Westerberg

FINALLY!

The problem was the Heroku memcachier add-on. That one is obviously mandatory. After adding it to my app all assets are served correctly!

Please add the following lines to the instructions:

Add Memcachier add-on to your app:
heroku addons:add memcachier

@mixirving: I've shared our installation notes document with you on google drive. Might be helpful for others as well.

I'm super happy this finally worked out. Now I can do some real work :)

Robert Guthrie

@elimisteve loomio.org runs on heroku. We have three 2X dynos which handle between 100 and 400 requests per minute with an average response time of 160ms.

Steve Phillips / @elimisteve

@robertguthrie Thanks for the infrastructure info. The 2x dynos handle 100-400 reqs/minute each or combined?

Robert Guthrie

combined.. but not fully utilised.