[it-squad] - about emails @pirateparty.be (mailboxes, forwards, mailing-lists)
The situation
Our mails are hosted at Infomaniak and it costs now 120€ / year. They invites us to upgrade because their old tools are evolving. It is not mandatory but for instance, the antispam filter is not working nicely.
Until now they provided a single fee for multiple services and they are evolving to a «on demand price/service».
(I suppose that one day or another we will have to upgrade or change)
We use them mainly for our emails @pirateparty.be.
The website, the wiki and some other old tools are hosted by Hetzner.
The Upgrade Proposal
Is to agree to the upgrade and optimize our usage but it will cost us probably more then 120€ / year.
- Infomaniak's mailboxes price calculator
- Follow this guide to do the migration.
The Do It Yourself Proposal
Is to implement our own mail server+webmail+imap+antispam+antivirus+backup and ask our current @piratepaty.be mail users to migrate their mailboxes or keep a copy for and by themslef. This may ask more human resources, time and discussions and probably not less money neither.
- For an it-squad it is a nice chalenge and probably a long discussion about the tools. :construction_worker_tone1:
The idea is to evolve to one of those two Proposals and find a concensus about it if possible.
Sandrine Debatty Wed 25 Jan 2017 6:21AM
Je suis ok avec ta proposition HgO.
Pascal Dk Thu 2 Mar 2017 5:50AM
Still the same team behind the project thou
(not only the founders, the whole team).
tierce (Thierry Fenasse) Fri 24 Mar 2017 9:24AM
English
Knowing that our IT costs where reduced by 50% (explained here),
Recieving this morning from Infomaniak, the information about the auto-upgrade on the 4th of April
Meaning that the new admin panel will probably be more confortable to use and manage, but the new costs may raise from 120€/year to ±350€ regarding the number of mailboxes we keep.
Having a small active it-squad.
The upgrade will be done automatically.
Français
Sachant que les frais informatiques viennent d'être réduits de moitier (expliqué ici),
Ayant reçu de Infomaniak l'information que la Console Évoluera le 4 avril
Ce qui veut dire que la migration vers leur nouveaux outils (plus confortable?), et leurs nouveaux tarifs (probablement passant de 120€ / ans à ±350€ en fonction du nombre de boîtes mails qu'on gardera).
N'ayant qu'une petite it-squad active.
La mise à jour se fera automatiquement.
Nederlands
(bijgestaan vertaalmachine)
Wetende dat IT-uitgaven worden 50% verminderd (hier uitgelegd),
Een Infomaniak bericht vanochtend aangekomen uit te leggen dat de console zal worden bijgewerkt, op 4 april
Dit betekent dat de nieuwe admin panel waarschijnlijk meer comfortabel te gebruiken en te beheren zal zijn, maar de nieuwe kosten kunnen verhogen van 120 € / jaar tot ±350 € met betrekking tot het aantal mailboxen we houden.
We hebben een kleine actieve it-squad.
De upgrade zal automatisch worden gedaan.
tierce (Thierry Fenasse) Mon 10 Apr 2017 1:14PM
If we (it-squad / Belgian Pirates) want, have time, ressources (more human than anyting else :sweat_smile: ) ... we can also set-up our own mail server.
Knowing that :
Setting up our own mail server is doable, but from personal experience is a real PITA:
* Microsoft mail services blacklist whole netblocks (Hetzner, OVH)
* Google is very picky and inconsistent (sometimes silently end up in the spam folder)
In addition to just installing the SMTP server, the following MUST be met to hope reaching the mailbox:
- Proper forward DNS records (A, AAAA) and reverse (PTR)
- Proper SPF records in the DNS
- Working DKIM setup (both on mail server and in DNS records)
- Bonus: DMARC records for reporting
- Bonus: TLS on mail server
- Proper MX record (mail reception only)
And we can use this bundle of FOSS tools : https://hub.docker.com/r/tvial/docker-mailserver/
tierce (Thierry Fenasse) Sun 21 May 2017 5:28PM
Because the IPs of Hetzner are blacklisted «by default» by Microsoft (as a anti-spam solution), once we have a new mail server hosted at Hetzner, and if we want to be able to send emails to live.be, outlook.com, hotmail.com and maybe everyone hosted on office365 ...
We also have to consider to have a Microsoft Live Account
- Do we create one for the IT-Squad?
Subscribing to Microsoft JMRP and SNDS to «register the IP address of our future mail server from Hetzner as a workaround for the Hetzner blocking by Microsoft.
Thanks to @gilbertotorres for the share.
In addition of what we wrote before
Setting up our own mail server is doable, but a real PITA (Pain In The Ass):
- Microsoft mail services blacklist whole netblocks (Hetzner, OVH)
- Google is very picky and inconsistent (sometimes silently end up in the spam folder)
In addition to just installing the SMTP server, the following MUST be met to hope reaching the mailbox:
- Proper forward DNS records (A, AAAA) and reverse (PTR)
- Proper SPF records in the DNS
- Working DKIM setup (both on mail server and in DNS records)
- Bonus: DMARC records for reporting
- Bonus: TLS on mail server
- Proper MX record (mail reception only)
As a reminder and for our Mastodon
We have had to register a Mailgun account to have a «clean smtp» to send mail (notifications, password recovery, ...).
The account was registered using those required informations:
- An email: we used the personnal mail of @damiensrobert
- A visa card: we used mine.
- A billing address: Mermaid vsw/asbl
Damiens ROBERT Tue 11 Apr 2017 8:22PM
Just a quick note that is important, https://hub.docker.com/r/tvial/docker-mailserver/ seems very complete, well made and very easy to install but the tools installed are command line tools. Meaning that to manage them we'll have to use SSH and the linux terminal.
I am sure there are web applications that allow you to configure the services installed by this project but integrating the web applications with this project will require work and testing.
Finally, we will have to configure our own webmail client (probably roundcube) to use the services installed by the project.
On other words, it seems very professional but it will require quite some work. I think it would be a good solution to avoid to depend on providers who can stop or change the condition of their services and really controlling our infrastructure.
Note that as it is a open source project and that it seems very popular with over 500K downloads in the docker hub (1). It means we can probably have some traction to have other open source project to integrate nicely with it. At least, we can try to open issues on the web applications bugtrackers that we will need to use to administer the mail not using the command line in order to ask them to support the docker mail server. They might answer positively and I would not bury the idea before trying.
ZeFredz Sun 16 Apr 2017 11:37AM
This docker image seems to have everything described in the 4 articles posted by @tierce
tierce (Thierry Fenasse) Sun 16 Apr 2017 11:27AM
Thanks to @zefredz : a serie of articles about running your own mail server :
- How to run your own e-mail server with your own domain, part 1
- Taking e-mail back, part 2: Arming your server with Postfix and Dovecot
- Taking e-mail back, part 3: Fortifying your box against spammers
- Taking e-mail back, part 4: The finale, with webmail & everything after
... Ya plus qu'a lire / faire / etc....
ZeFredz Sun 16 Apr 2017 3:15PM
We (@tierce, @damiensrobert, @iljabaert and I) decided to install a test mail server under the domain parley.be (a domain we already have) using docker and the docker-mailserver fullstack.
At the end of the day, we have a partly working mail server (imap, smtp, tls) :)
What's left to do for next time : understand why ssl does not work properly (certificate error) and why clamav is failing to scan emails.
ZeFredz Sun 16 Apr 2017 3:20PM
we also have some issues with gmail and yahoo which are putting the emails from parley.be in the spam
tierce (Thierry Fenasse) Sun 16 Apr 2017 4:02PM
Regarding those the must have we forget to setup a PTR record ... now it's done.
I used mail.parley.be ... but I do not know if parley.be is better or not.
HgO Wed 26 Apr 2017 1:22PM
I know we are going towards our own mail server, but for your information yulPa (formerly Web4all) propose now a stand-alone mail service : https://yulpa.io/messagerie-zimbra-ne-mutu.html :)
Gilberto Torres Fri 19 May 2017 4:30PM
@5euro per account... WE can do it better ;)
HgO Fri 19 May 2017 5:05PM
That's not what I understand ' It's 5€ HT for a mail service with 25Go storage space.
Anyways, I guess we will discuss all that on Sunday (see https://wiki.pirateparty.be/ITSquad/Meeting/21_05_2017). We'll be glad to see you there ;)
Gilberto Torres Sun 21 May 2017 6:12PM
Hey everyone reading,
The "pita" mentioned on the previous post is a mecessary evil on every mail server running...
Best regards,Gilberto Torres
tierce (Thierry Fenasse) Mon 5 Jun 2017 8:54AM
During our latest IT meeting we have installed a YuNoHost server under our parley.be domain name.
The main goal (correct me if I'm wrong) is to replace the Infomaniak provider as an SMTP server to
- send our Newsletters (like the last one nl, fr, en),
- send notifications / lost password from our Wiki,
- send notifications / lost password from our WordPress,
- send notifications / lost password from our Mastodon,
We are happy with a 10/10 result :fireworks: regarding some of the PITA/basic requierments to avoid being considered as spammers when we send mails.
Another step (requiring some contracts to be signed) is to add our mail server IP address to the Microsoft JMRP and SNDS. I do not know if it will be necessary to do the same for Google, Yahoo, Apple or any other of those corporation networks. :robot:
A secondary goal may be to replace Infomaniak for our users having a pirateparty.be mailbox.
Patrick Installé Mon 5 Jun 2017 9:33AM
Great job. I love your feedbacks. That's the pirates way of life.
HgO Tue 25 Jul 2017 9:02AM
So, I made this diagram to help me have a better view on what is the current situation in this mail mess ! :) I also made another diagram to show you what I propose as refactoring. You can see on the bottom-left things I wamt to remove ;)
I'm not sure about:
* treasure: should it be connected to finance, or a standalone ?
* merchandising and shop: do we keep them ? purpose ? connected to finance ?
* legal: purpose ?!
* having a mail per web service: overkill ?
* postmaster: what's the purpose ? difference with webmaster ?
* ahoy: should it stay empty or not ?
* get-me-in: does anybody knows what it was used for ????
Questions, remarks, objections, etc. ? :D Thanks !
Renaud Van Eeckhout Tue 25 Jul 2017 10:19AM
I think the get-me-in was the form that people could fill on the Drupal website.
Shop I believe was for the webstore we had.
The "web" is a new mail you propose?
I'd also delete legal (if no particular reason to keep it) and coreteam.
HgO Tue 25 Jul 2017 10:42AM
Thanks ! I didn't even know we had a shop :o No, "web" exists yet, but as you can see it forwards to only one guy, and I'm not sure he is in the party anymore...
I'm ok for deleting "legal", but I would keep coreteam for now as it is used on our DNS details :(
tierce (Thierry Fenasse) Tue 25 Jul 2017 12:23PM
That's a clear diagram! Thank you soo much for having done this!
Paul Bossu Thu 3 Aug 2017 7:39AM
shop and merchandising was indeed for all the promotion kit.. originally on the website (drupal jurgen then wordpress marouan, then.drupal Tom, ... ) all the stock was in à box at the office but it seem we lost it when the German.pirates closed the office
Legal was an.idea of marouan for all the legal stuff (asbl but also laws proposal)
HgO Thu 3 Aug 2017 8:41AM
Thanks @paulbossu ! :) Yes, I was there when we were supposed to get back the merchandising, but the office was clean and empty... :( I think @pascaldk tried to contact them, but I guess it didn't work ?
I've removed mail accounts of those who gave me their agreement so far, and I've updated the current schema, you can see it on the wiki (because there is a versioning system there) : https://wiki.pirateparty.be/File:Emails_Structure.png :)
Paul Bossu Thu 3 Aug 2017 10:01AM
Je dois encore regarder pour paul@pirateparty
HgO Mon 28 Aug 2017 1:01PM
Sooo a quick update :
* We switched to the new console, which means that everything is pretty but will be more expensive (as this was expected)
* We have till 17th January 2018 to reduce the number of pirate mail accounts and mailing lists (because our current contract ends at this date)
* On the 50 personal accounts, I received 13 "yes I want to keep my pirate address" and 19 "no, you can remove it". I don't know for the 18 remaining people...
* In the new console, if I want to renew the contract now (to see how much it would cost us), they say "450€ HT", while it should be 410€ TTC (at the moment). I suppose this is a bug, meaning that we will have to contact infomaniak...
Also, I will do the re-structuration of the mailing lists (see the schemas above) probably in September / October :)
@paulbossu For now, I'm keeping your mail address, so you have all the time you want to decide whether you want to keep it or not ;) As I told you, it's not a problem to keep a few "uncertain" pirate address. The problem was that we had dozens of old and unused pirate accounts waiting to be removed :p
HgO Mon 16 Oct 2017 10:29AM
I removed the pirate e-mails account of those who never answered my "warnings".
As the address gent@ppbe was left empty after the removals, I decided to delete this account... Let me know if this was a bad idea :sweat_smile:
In the end, we have 63 e-mails remaining. This should costs us 387€/year. Next step is to restructure the e-mails (see the schema above). Ideally, I would like to have an ITSquad meeting in order to work on this and to think further on this new structure. I think that most addresses can be "merged" through aliases.
We should also contact infomaniak in order to have some clarifications on their pricings...
HgO Mon 23 Oct 2017 9:16PM
Sooo @tierce and I contacted infomaniak in order to clarify their pricings stuff. It appears that due to a kind of "legal bug", we will keep the old pricing, that is 120€/year regardless of the number of mails :confetti_ball: :tada:
For the details, they cannot force us to switch to their new pricing. So, as long as we keep renewing our contract each year, we will keep benefiting of this "special" pricing (cc @patrickinstalle ).
This also means that we have a web hosting that we could use (we don't use it yet) ;)
Finally, I think that the mailing lists still need a better structure, and I hope that we will be able to work on that during the incoming ITSquad meeting (see this poll for deciding on the date).
HgO Sat 4 Nov 2017 5:08PM
Ok, I've made several changes in the e-mail structure. Here isthe result :
The most important thing is that all web services are now aliases of web@ppbe. Thus, this mailbox can be used for password recovery. I also changed passwords for most of those services (with the help of @pascaldk) and stored them in a place where ITSquad can find the credentials easily (on our encrypted tiddlywiki, to be precise).
I removed a lot of mailing lists, too (legal, bin, listener, agenda, get-me-in, wikileaks, web4all, lqfb, opinion, ixelles, etc.) !
I also removed the redirections on newsletter@ppbe, and now people who send a mail to this address will receive an automatic message, telling them to contact contact@ppbe instead :)
Now, I'm unsure about the following mailing list :
* info@ppbe and international@ppbe : They seem to collect tons of spam, and I tempted to delete them. I think that they are useless, since we have already contact@ppbe.
* redactie@ppbe Not sure what's the purpose of this one, and it collects also a lot of spam... I'd remove this one.
* shop@ppbe and merchandising@ppbe I'd like to remove these mail accounts, but there seems to have some information about previous orders, Is this information important ? I'm afraid to lose a bit of our memory here, but maybe I'm too careful.
* coreteam@ppbe Aha the funny one :) So, we should remove it, but I think our DNS, bills, and other technical stuff are linked to this mail address...
Valerie D. Sun 5 Nov 2017 9:05PM
@hgo redactie was the writing squad, for articles on our website, knack and co.
HgO Tue 7 Nov 2017 1:46PM
Then I guess I can delete this mailing list, as we use loomio at the moment ? If we need a mail address for this purpose, we will just have to create it again.
HgO · Tue 24 Jan 2017 9:30PM
Oui, y compris celles des crews (surtout celles des crews en fait, car je pense qu'il y en a très peu d'actives). Quand je parle des mailing lists, ce sont toutes les adresses mails @ppbe qui ne sont ni des alias, ni des adresses personnelles.