Poll does not close after that expiration time ends. Bug or my fault?

Hi @Giuseppe. I would first check that your crontab is setup and running. It's an hourly job to close polls.

Hey @Rob Guthrie , thanks for your hint.

I checked with the command "ps -ef | grep crond".
And this is what I get:

ps -ef |grep crond 
gmt       8797  8782  0 18:36 pts/0    00:00:00 grep crond

So for me it seems, that the process is running, right?
This is what is inside the cronetab file when I run"crontab -e":

0 * * * * /snap/bin/docker exec loomio-worker bundle exec rake loomio:hourly_tasks > ~/rake.log 2>&10 
0 * * * ~/loomio-deploy/scripts/create_backup ~/loomio-deploy > ~/backup.log 2>&1; s3cmd put ~/loomio-deploy/back$
0 4 * * * /sbin/shutdown -r +5

Is there something else that I could check?

By the way, I noticed that I do not get any mail that is showing "What happend on Loomio" nor any reminder mail that a due date is beeing to be reached.

Is this a hint that something is not wirking with the crontab package?

Okay. I noticed that by using "crontab -e" the user's crontab file is opened. I got crontab to work by adding the commands into the root crontab file in etc/ folder. There, the cron commands seem to work. I was able to make an automated, nightly reboot and automated nightly backup thorugh the create_backup script.

But still I was not able to get the loomio hourly tasks to run. I noticed, that the folder "/snap/bin/docker" is not available on my system - like in the command in the readme file. Is this normal? What I have is the path "/snap/docker" , but the command does not work with this.

In the "rake.log" file that is created in the cronetab, the following message occurs. Does that help?

Error: No such container: loomio-worker

Hello. I do not know why - but it see,s to work now. Maybe because of some restarts but I get emails and the proposals are ended after the expriation date is reached.

I have only one missing issue - I am getting now every our a notification of a proposal that reached the limit. But I deleted the discission for that. How can I deactivate or find this proposal manually to delete it or deactivate it? Is there a possibility?

Sorry, can you please explain that again for me?

You're getting a proposal closed notification via email each hour? For a poll with the same id/URL each time?

Yes exactly! Like you said.
But in detail, to make it clear - The notification is saying that the poll "ended", not that it was closed.

I did the following steps this morning:

I looked inside the database via psql and I searched in the polls table for the value in the "closing at" and the "closed at" column.

I noticed, that there was a value existend as a date inside the "closing at" cell, but not inside the "closed at" cell - it was empty.

I put manually a value inside, that is the same date as inside "closed at".

I will see, if this is now working and fixed.

I have no idea, how this could happen...
Maybe a reason could be, because the crontab was not working in the past and I "deleted" the proposal without closing it before. So the closed at value was never set. And now, where the crontab is working, it is giving me the notifications.

Could that be?

The cronjob is supposed to fill the closed_at value when it actually closes the poll.

you could run rake loomio:hourly_tasks and see what the output is to learn more about what's happening.

I ran rake like this:

sudo docker-compose run app rake loomio:hourly_tasks > ~/rake_loomio.log

There are no messages logged.

sudo /snap/bin/docker exec loomio-worker bundle exec rake loomio:hourly_tasks > ~/rake_loomio.log

The command above does not create any outputs as well.

Can you repeat what issue you are trying to solve please?

Sure.

- The proposals and polls are not ending automatically.

- No daily "what happend on loomio" is arriving".

I am not sure if my crontab configuration is set up well. If I understood it right, there is a difference if I run crontab -e or sudo crontab -e.

Every of my docker-compose commands, I have to run them as sudo , so that they work.

This is how my crontab look like, when I run crontab -e:

# Edit this file to introduce tasks to be run by cron.
#
# Each task to run has to be defined through a single line
# indicating with different fields when the task will be run
# and what command to run for the task
#
# To define the time you can provide concrete values for
# minute (m), hour (h), day of month (dom), month (mon),
# and day of week (dow) or use '*' in these fields (for 'any').#
# Notice that tasks will be started based on the cron's system
# daemon's notion of time and timezones.
#
# Output of the crontab jobs (including errors) is sent through
# email to the user the crontab file belongs to (unless redirected).
#
# For example, you can run a backup of all your user accounts
# at 5 a.m every week with:
# 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/
#
# For more information see the manual pages of crontab(5) and cron(8)
#
# m h  dom mon dow   command
#
0 * * * *  root /snap/bin/docker exec loomio-worker bundle exec rake loomio:hourly_tasks > ~/rake_crontab.log 2>&1

0 0 * * *  ~/loomio-deploy/scripts/create_backup ~/loomio-deploy > ~/backup_crontab.log 2>&1

... the sudo crontab -e has no added lines.

The logfile that I get after my crontab -e is executed remains empty.

When I run the command manually as sudo with the command line sudo /snap/bin/docker exec loomio-worker bundle exec rake loomio:hourly_tasks > ~/rake.log 2>&1 , I get the following log:

opened
starting SSL for www.loomio.org:443...
SSL established, protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384
<- "POST /api/v1/usage_reports HTTP/1.1\r\nContent-Type: application/json; charset=utf-8\r\nAccept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\nAccept: */*\r\nUser-Agent: Ruby\r\nC$
<- "{\"usage_report\":{\"version\":\"2.1.9\",\"groups_count\":9,\"users_count\":38,\"discussions_count\":26,\"comments_count\":64,\"polls_count\":28,\"stances_count\":107,\"visits_count\":$
-> "HTTP/1.1 200 OK\r\n"
-> "Date: Tue, 05 May 2020 00:19:01 GMT\r\n"
-> "Content-Type: application/json\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Connection: close\r\n"
-> "Set-Cookie: __cfduid=d37944167e3c80408adb122b0bd6960f01588637940; expires=Thu, 04-Jun-20 00:19:00 GMT; path=/; domain=.loomio.org; HttpOnly; SameSite=Lax\r\n"
-> "X-Frame-Options: SAMEORIGIN\r\n"
-> "X-Xss-Protection: 1; mode=block\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "X-Download-Options: noopen\r\n"
-> "X-Permitted-Cross-Domain-Policies: none\r\n"
-> "Referrer-Policy: strict-origin-when-cross-origin\r\n"
-> "Cache-Control: no-cache\r\n"
-> "Set-Cookie: ahoy_visitor=0f4dc20b-69a0-47f9-b168-4536b9de7596; path=/; expires=Thu, 05 May 2022 00:19:01 -0000\r\n"
-> "Set-Cookie: ahoy_visit=2ce0a3c7-0174-45c9-9fa7-28355970be41; path=/; expires=Tue, 05 May 2020 00:49:01 -0000\r\n"
-> "Set-Cookie: CSRF-TOKEN=XLuZ2wypDwBMozDIjTWu81iIM8n13GDUl8MauGx5ArshfrThRKaJZU%2Bqh89nK3MtOe2tdOSWzbaVT7TfEOIHIA%3D%3D; path=/\r\n"
-> "Set-Cookie: _loomio=emFSMEtoaWloNTg1Y1FGQ2pucWZXamJrSkJHMms3ZFRBZ0NHK0tZbUp3RzRSOGtaSm5wamt1U0lwdHl6Unl5ZnAyN0t3eksxNUphbTlSY3o3R3ZlVFRNUWFoSjduTFBTK0JlOVNuUVd4T2IzNUZzVWJYZHZycENWUHY1$
-> "X-Request-Id: f2b65e02-3ccf-4245-a9d5-ff824abf4421\r\n"
-> "X-Runtime: 0.019164\r\n"
-> "Vary: Accept-Encoding\r\n"
-> "Content-Encoding: gzip\r\n"
-> "Via: 1.1 vegur\r\n"
-> "CF-Cache-Status: DYNAMIC\r\n"
-> "Expect-CT: max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"\r\n"
-> "Server: cloudflare\r\n"
-> "CF-RAY: 58e64998ad330746-FRA\r\n"
-> "alt-svc: h3-27=\":443\"; ma=86400, h3-25=\":443\"; ma=86400, h3-24=\":443\"; ma=86400, h3-23=\":443\"; ma=86400\r\n"
-> "cf-request-id: 0283cc536800000746e9228200000001\r\n"
-> "\r\n"
-> "19\r\n"
reading 25 bytes...
-> "\x1F\x8B\b\x00\xF5\xB0\xB0^\x00\x03\x00\x00\x00\xFF\xFF\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00"
read 25 bytes
reading 2 bytes...
-> "\r\n"
read 2 bytes
-> "0\r\n"
-> "\r\n"
Conn close

Should I use crontab -e or sudo crontab -e ?

It looks to me that any tasks that are background jobs are not executing.

If you load https://yourserver.com/sidekiq it will show you the status of background jobs, and might explain what's going on in there.

You'll need to be logged in as an admin user.

Oh cool! I did not know this feature. This is great!

I made a screnshot and I placed it here. (unfortunately, it is in german, but I clicked on the tab "Tries".



It seems that for some jobs there is a SMTP failure. Could that be?

I just tried my SMTP through the login via email verification: So it seems to work - I received an email from the server with the verification code. Furthermore I receive also emails, when e.g. another person is answering on one of my comments. That is also a hint, that SMTP ist working, isn't it?

By the way, some days ago everything seemed to work fine. The polls where claused automatically and I got a reminder mail before the poll closed. I cannot explain why it worked suddenly. Now, it does not work anymore... I think that I changed nothing :/ And I cannot explain myself why..

ok! so it seems like it is retrying the expire lapsed polls job because an email is not able to send correctly. The job itself should probably be rewritten so that it does not do this, but you can still fix this by either deleting the jobs, and/or, working out why the email's are failing.

Okay. I deleted the jobs through the sidekiq applicaton.

Should I delte also the lines that are listed in the queue tab?

There is one thing that I did not unserstand yet concerning the crontab lines.

Is an empty log file good or bad?

I deleted the entries in the qaiting queue now. Let's see what will happen.

I looked now also in the details of the Failure Messege. This is what is logged inside - but I cannot figure it out.

I see that there is a "block in deliver mail" sentences inside the log.

I checked the SMTP Server, that is from the same provider, where the DNS is running.

Error
Error Class Net: SMTPFatalError
Error Message: 550-Requested action not taken: mailbox unavailable
Error Report

/usr/local/lib/ruby/2.6.0/net/smtp.rb:969:in `check_response' /usr/local/lib/ruby/2.6.0/net/smtp.rb:937:in`getok'  
/usr/local/lib/ruby/2.6.0/net/smtp.rb:865:in `rcptto' /usr/local/lib/ruby/2.6.0/net/smtp.rb:846:in`block in rcptto\_list'  
/usr/local/lib/ruby/2.6.0/net/smtp.rb:844:in `each' /usr/local/lib/ruby/2.6.0/net/smtp.rb:844:in`rcptto\_list'  
/usr/local/lib/ruby/2.6.0/net/smtp.rb:659:in `send_message' /usr/local/bundle/gems/mail-2.7.1/lib/mail/network/delivery_methods/smtp_connection.rb:54:in`deliver!'  
/usr/local/bundle/gems/mail-2.7.1/lib/mail/network/delivery\_methods/smtp.rb:101:in `block in deliver!' /usr/local/lib/ruby/2.6.0/net/smtp.rb:519:in`start'  
/usr/local/bundle/gems/mail-2.7.1/lib/mail/network/delivery\_methods/smtp.rb:109:in `start_smtp_session' /usr/local/bundle/gems/mail-2.7.1/lib/mail/network/delivery_methods/smtp.rb💯in`deliver!'  
/usr/local/bundle/gems/mail-2.7.1/lib/mail/message.rb:2159:in `do_delivery' /usr/local/bundle/gems/mail-2.7.1/lib/mail/message.rb:260:in`block in deliver'  
/usr/local/bundle/gems/actionmailer-5.2.4.2/lib/action\_mailer/base.rb:560:in `block in deliver_mail' /usr/local/bundle/gems/activesupport-5.2.4.2/lib/active_support/notifications.rb:168:in`block in instrument'  
/usr/local/bundle/gems/activesupport-5.2.4.2/lib/active\_support/notifications/instrumenter.rb:23:in `instrument' /usr/local/bundle/gems/activesupport-5.2.4.2/lib/active_support/notifications.rb:168:in`instrument'  
/usr/local/bundle/gems/actionmailer-5.2.4.2/lib/action\_mailer/base.rb:558:in `deliver_mail' /usr/local/bundle/gems/mail-2.7.1/lib/mail/message.rb:260:in`deliver'  
/usr/local/bundle/gems/actionmailer-5.2.4.2/lib/action\_mailer/message\_delivery.rb:114:in `block in deliver_now' /usr/local/bundle/gems/actionmailer-5.2.4.2/lib/action_mailer/rescuable.rb:17:in`handle\_exceptions'  
/usr/local/bundle/gems/actionmailer-5.2.4.2/lib/action\_mailer/message\_delivery.rb:113:in `deliver_now' /loomio/app/models/concerns/events/notify/author.rb:8:in`email\_author!'  
/loomio/app/models/events/poll\_expired.rb:16:in `email_author!' /loomio/app/models/concerns/events/notify/author.rb:4:in`trigger!'  
/loomio/app/models/concerns/events/notify/third\_party.rb:3:in `trigger!' /loomio/app/models/concerns/events/notify/in_app.rb:5:in`trigger!'  
/loomio/app/models/event.rb:45:in `tap' /loomio/app/models/event.rb:45:in`publish!'  
/loomio/app/models/events/poll\_expired.rb:7:in `publish!' /loomio/app/services/poll_service.rb:78:in`block in expire\_lapsed\_polls'  
/usr/local/bundle/gems/activerecord-5.2.4.2/lib/active\_record/relation/delegation.rb:71:in `each' /usr/local/bundle/gems/activerecord-5.2.4.2/lib/active_record/relation/delegation.rb:71:in`each'  
/loomio/app/services/poll\_service.rb:75:in `expire_lapsed_polls' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/extensions/class_methods.rb:19:in`perform'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:196:in `execute_job' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:164:in`block (2 levels) in process'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/middleware/chain.rb:138:in `block in invoke' /usr/local/bundle/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/instrumentation/sidekiq.rb:32:in`block in call'  
/usr/local/bundle/gems/newrelic\_rpm-6.10.0.364/lib/new\_relic/agent/instrumentation/controller\_instrumentation.rb:376:in `perform_action_with_newrelic_trace' /usr/local/bundle/gems/newrelic_rpm-6.10.0.364/lib/new_relic/agent/instrumentation/sidekiq.rb:28:in`call'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/middleware/chain.rb:140:in `block in invoke' /usr/local/bundle/gems/sentry-raven-2.13.0/lib/raven/integrations/sidekiq.rb:9:in`call'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/middleware/chain.rb:140:in `block in invoke' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/middleware/chain.rb:143:in`invoke'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:163:in `block in process' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:136:in`block (6 levels) in dispatch'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/job\_retry.rb:111:in `local' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:135:in`block (5 levels) in dispatch'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/rails.rb:43:in `block in call' /usr/local/bundle/gems/activesupport-5.2.4.2/lib/active_support/execution_wrapper.rb:87:in`wrap'  
/usr/local/bundle/gems/activesupport-5.2.4.2/lib/active\_support/reloader.rb:73:in `block in wrap' /usr/local/bundle/gems/activesupport-5.2.4.2/lib/active_support/execution_wrapper.rb:87:in`wrap'  
/usr/local/bundle/gems/activesupport-5.2.4.2/lib/active\_support/reloader.rb:72:in `wrap' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/rails.rb:42:in`call'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:257:in`stats'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/job_logger.rb:13:in`call'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/job_retry.rb:78:in`global'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:124:in `block in dispatch' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/logger.rb:10:in`with'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/job\_logger.rb:33:in `prepare' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:123:in`dispatch'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:162:in `process' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:78:in`process\_one'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/processor.rb:68:in `run' /usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/util.rb:15:in`watchdog'  
/usr/local/bundle/gems/sidekiq-6.0.7/lib/sidekiq/util.rb:24:in \`block in safe\_thread'

What happens when the email is to a completely different address, for instance, a gmail address? Does it deliver?

I did not understand your question. How can I check to which mail address the mail is delivered.

I am receiving mails to my Gmail address, with which I created my Admin account.

I created a new group on the loomio server, then a new discussion with a poll wirth experation date.

The service worked, I got an email that the poll was closed. I checked in the loomio discussion and the state of the poll changed to ended.

So here it worked. But I did not get a what happend on loomio mail.

In sidekiq, there are still occurring error messages of the same kind.

The catch up email is only sent if there is content which you have not read, is it possible that you have just read all the content, that's why you don't get any mail?

The error you're seeing must be to another user with a bad email address. And it could be "Loomio Bot", which I should remove. That failure won't stop other emails from being delivered.

Oh yes. You were right. I did no Catch Up Mail because I was very active in the discussions and had nothing unread. I tested it and I can confirm that I am receiving this mails.

Maybe, like you assume, the Loomio Bot is the cause of the error messages in the Sidekiq. How can I check this to get this information for you?

Thanks four help, @Rob Guthrie