I’m tempted to move to email with my own domain, but don’t really care about the server side of things, I guess I’m following the line that self hosted email is not easy to do, and not sure how reliable I can make my servers.
The other option is to have some other system do the email hosting, and if it is through my domain then there should be no issues with moving around if needed. Hosting all the email archive isn’t really needed either, I’d be fine self hosting or storing the archive.
Is there a set up where some remote service handles email exchange, and you just sync with it eg through POP style interfaces? Id actually prefer if my own server was doing the sync rather than a desktop running Thunderbird etc, that way it is kind of self hosted but also robust to down time. What’s the best practice along these lines?
POP isn’t sync; pop downloads all the messages locally and you tell the server what to do with them once they’re downloaded.
The result is that you have your messages on your device. But when you go to another device, it’s a different set of messages with different read status.
Unless something has changed, that’s how it worked last time I used POP and that’s why everyone used IMAP now that there’s multiple devices (phones, laptops) in play.
You’re right about self hosting email; it’s a pain.
I meant IMAP really, just got the terms mixed up, thanks
FWIW, from what I read, the difficulty in selfhosting email is mainly in how to not get your outgoing mail insta-plonked by increasingly fussy and moody spam filters. Shows up that a huge percentage of receivers are now on gmail, trailed by a few other major players like M$, and you’re on their whimsy, with no way to appeal plonk decisions (and challenging to even find out if they plonked or not). This was reported so even before the era of “AI”, I wouldn’t be surprised if it were even harder now.
Sounds almost like the corpos have a monopoly on email.
Not entirely true. I do selfhost my Email Server however I do it for 20 years so the IP associated has excellent reputation due to the long record of good behavior. So it is possible but it is an uphill struggle.
Yep, that’s exactly the sentiment I saw in people’s writing, that email has become almost centralized in practice, even though in theory it’s not. I saw this also generalized, that even successfuly promoted distributed/federated protocols still face the constant risk and challenge of becoming centralized in practice, when a major player manages to capture the majority of the field.
AFAIU, even Mastodon community needs to (and does) purposefully work on promoting alternative instances, so that not everybody ends up on mastodon.social…
IMAP is the protocol you probably want.
I’m quite happy with “Mail In A Box” as a platform for hosting mail locally with clients.
There are companies that will do inbound spam filtering and outbound mail relaying to ensure high reputation for outgoing main also. I have clients on duocircle but they’re grandfathered in on reasonably priced plans (the new plans are probably a little expensive for self hosted home mail)
Or JMAP, which is IMAP that doesn’t suck.
How so?
Its basically IMAP, WebDAV, and CalDAV, plus a few other things, in one protocol. It greatly simplifies and unifies communications over a modern HTTPS connection while being an open standard.
See JMAP Destroy All Monsters by the protocol author for an intro.
POP and IMAP are still alive.
Services are currently drop POP3 support. Gmail and Microsoft among them. I would consider it obsolete.
IMAP is very much alive but has so many pain points that JMAP was created to replace it, along with CardDAV and CalDAV for calendar and contacts.
Fastmail created and uses JMAP and Stalwart has first-class support for it.
Can you imagine how many fucks I give about what “services” like Microsoft drops?
The difficulty of self hosting your own email is high, but if you aim to do it, i would recommend Stalwart for self hosting. POP is not what you want as other people are saying. There is also a newer protocal to IMAP called JMAP.
Some things to consider are your inbound and outbound mail paths.
For incoming: Do you have a VPS to be your entry point? Or do you plan to port forward (assuming you even can given the prevalence of CGNAT)? There is also a way to teach a web worker like the kind cloudflare let’s you use for free to break the email down and POST it to a listener in your server across a tunnel and the listener can put it back together into an email at your server and drop it off. That being the only way I have found to get inbound mail behind a double NAT (CGNAT) without a VPS.
For outbound, if you send stuff to major SMTP providers from a residential IP they will block it or mark as spam pretty heavily. You may want to consider a relay to make sure people actually see what you send. Brevo has a free tier, SMTP2Go is solid. There are others as well.
There are other considerations like how will the end user interact with the mail server? I host a Bulwark instance and really love what they are doing with the project, but there are quite a few options for that.
If you end up giving it a go, good luck, and know that it is possible!
I’ve taken the second approach you mentioned: I own the domain, host my email with a service provider (I use Zoho Mail), and manage my DNS through Cloudflare. Here are three insights I’ve gained from this experience.
First, by managing your email provider and domain DNS separately, if you ever want to switch providers in the future, you’ll only need to update the MX and a few TXT records. Your email address will remain the same, and others won’t even notice.
Second, set up SPF, DKIM, and DMARC right from the start. Be sure to enter the recipient address for DMARC aggregate reports so you can receive daily reports showing which servers are sending emails using your domain and whether they’ve passed verification.
Third, even if a new domain passes all three checks, emails sent to Outlook may still end up in the spam folder at first. This is because the new domain lacks a reputation. It’s not a configuration error, and it takes time to gradually build that reputation.
You can use
imapsyncto keep a local synced copy your email in maildir format. That, along with a domain you own, solves the “I own my own email sycned to my server” problem pretty much completely. Switching email providers is just syncing to the new server (whichimapsyncsupports) and updating your MX records on your domain.Some email clients (like
aerc) can just operate on the maildir directly. That’s what I do, but there are lots of ways to do this.One option is to just keep using your email client the way you are now talking directly to your mailserver, and just rely on the
imapsyncside to be your owned backup/migration path.Alternatively you can run your own local IMAP or POP server like Dovecot backed by the local maildir. So that’s
imapsync -> dovecot -> thunderbird (or similar). It’s a little more software to set up, but once it’s set up it should be very reliable and gives you total control over what email client you use and full ownership of your data.Thanks, that sounds useful.
Do you know of a self hosted webmail interface that has good features? So I guess that would run on the self hosted mail archive after imapsync.
I don’t have any personal experience with self hosted webmail. I know there are a bunch out there and definitely both options that can sit in front of a maildir and options that can sit in front of a POP/IMAP server like Dovecot.
I did a little googling, and if I wanted to set this up, the first thing I’d check out is something like
imapsync -> dovecot -> Roundcube.Dovecot provides a really reliable maildir -> IMAP/POP conversion layer, and once you’ve got that, most things should work.
You can do what you want with IMAP too.
I use isync, it’s a CLI tool that can sync between IMAP accounts or between IMAP and local plain storage and it has the option to delete messages over a certain age after it copies them.
Is there a set up where some remote service handles email exchange, and you just sync with it
Any regular email provider can do this as long as it offers IMAP. I use email clients with my provider’s IMAP but I also use isync on the server to periodically pull all email locally and delete messages older than 30 days. I’ve also installed a local private IMAP server and a private webmail client on top of that private IMAP, which I use to search through old emails.
This way I normally use the “live” mailbox with my PC/mobile devices, and use the private webmail when I need to search for anything over 30 days old.
What I do is not “selfhosted email”, technically speaking, because I rely on the live IMAP and SMTP from a provider. But I can switch provider at any time because all my email is @ my custom domains, so I can simply point my DNS to another provider and update my apps to use the new provider’s IMAP/SMTP server.
Yes that sounds like what I’m trying to set up
It depends what you mean by sync. There are plenty of services that will host your email server and you just connect via IMAP from all the clients.
But if you mean sync to an archive, look into openarchiver.
Yes I want to sync to an archive, sounds useful
I’ve been using it for about a year. Archiving three different mailboxes. So far so good.
What about JMAP, guys? Is that cool?
One of the reasons I decided to self host email was specifically to use jmap.
Really! Tell me more about it, if you want! I’m excited to learn. 😊
This page has a good overview. Its smaller, it’s faster, it makes more sense to implement, it’s easier to communicate with. POP3 and IMAP are really old. People don’t here much about it because big email providers have no reason to change or improve. For instance, many can’t handle IPV6 still.
It’s basically IMAP but using JSON. That’s the J in JMAP.
Pop is download only (and optionally delete on server when downloaded), IMAP is sync so read states, moves, deletes, etc., will sync to the server and between all clients.
But both exist still, just most email providers want you to leave your email on their servers so they can charge you for storage, scan it for advertising, etc., and most people want the state to sync across devices, so a lot of places just don’t implement it anymore and just allow either web-only or IMAP-only (usually paid since they don’t get to show you ads).
And yes you can set up a local software to download email from another server offering POP or IMAP and then either use a client application or self hosted web application to view them. This is not too common so it might be harder to find instructions, but it’s doable.
Which part are you concerned with. The biggest issue with email servers is trust. Many companies will refuse to connect to your SMTP server to send you emails or reject your emails your SMTP server is sending to them if you aren’t on a whitelist or if your IP address has been used for spamming in the past and it’s on a black list. The easiest way to get around this is to subscribe to an SMTP proxy service.
The other issue you may find will happen no matter if you selfhost or not. When using a custom domain, some servers have whitelists of domains in addition to IP addresses and if you aren’t on it they will silently reject emails you send or that they were supposed to send to you. I have a couple of places (health care and financial companies mostly) who do this, so I have to use gmail, which ironically means the email is less secure, especially with Google’s recent changes in policy about reading your emails.
Yes I was looking into options like this because of the server trust issues, I haven’t looked into SMTP proxies and that could be an option, or maybe it is the same thing. Basically just an intermediate host that has established trust, which I can sync from as needed, while also being robust to downtime on my end
SMTP proxy will have that trust for you and help you on setting up things at the domain level like reverse DNS, DKIM, etc., that are needed to combat forged “from” addresses and many selfhosters miss.
Only difference is you need a full email stack on your side. Only SMTP traffic (in and out) would go through the proxy and to your local SMTP server. It solves the IP trust issue in many cases (though not all as some places do block the proxies on purpose as well), but nothing can really solve the domain trust issues when they come up except directly contacting the companies who have those policies and often resolving it isn’t really possible since they often don’t have external tech support for that stuff. It’s so annoying when they only whitelist gmail, outlook, and a few other big company domains.
There are then various ways to view the emails and compose and send. If you want to have clients accessing the email via IMAP/POP then you need that server in addition to the SMTP server, but all of that traffic could then be kept local. If you want to only have a web client there are clients that access the email directly without need for IMAP/POP.
But this way, although your email does pass through the proxy so you need to trust them, at least the data at rest is not stored there which increase the likelihood they’ll change policies on reading your email like gmail did, or charge you for storage, which is expensive these days.
Personally I use mailcow which is a pretty nice stack, but it would probably be overkill for many people. And I have a good understanding of the issues with both IP and domain white-/black-listing. So it works decently for my needs.
I did this a few years back and chose to host, for now, with migadu. They have imap, pop, and a web interface. I use POP myself, because I prefer to house my data.
Client-wise, I use mutt and download messages with mpop, which is a snap to compile if your distro doesn’t have a package.
I have no complaints. This setup is flexible and simple. Migadu provides clear instructions for setting up your domain.
Good point, I assumed email was asynchronous on the client side but needed to be online on the server side. If it is robust to server down time that makes it easier
