Just this last weekend I started moving clients off of the mail hosting services I've managed for years and onto Fastmail. I have self-hosted my email since around 2005, and had my first email address around 1994 or thereabouts. It has never been more broken than it is now, and I mean that without even the slightest hyperbole.
The first RBL showed up around 1998, operated by Paul Vixie of DNS fame. I was using a small ISP in the East Bay at the time and they hated the MAPS RBL, especially the idea that they had to deal with some third party to resolve mail transport issues. But, from 1998 until the early 2000s, RBLs proliferated and lots of mail services subscribed to one or more. The one thing they had going for them was that they were easy to query, so it took no effort to find out if you were on the list, and if you were, there was usually a living, breathing human being somewhere that you could contact to get the matter resolved. It was a pain in the butt, but it could be handled.
During this time, the responsibility for ensuring that a message was received shifted from the recipient to the sender. Beleaguered systems administrators soon found themselves in a situation where they were supposed to be responsible for both ensuring that no spam reached their customers and that all of their customers' email reached the intended recipients.
Gmail came along and decided that, because they were operating "at scale", they didn't need to play in the same ecosystem. Over the years, ensuring that a message lands in a Gmail user's inbox has turned in to an infuriating game of trial-and-error. Gmail can do this because they now manage between 40% and 60% of the internet's email traffic.
AT&T/SBCGlobal/Yahoo/whoever they are now seem to have recently penalized all of Linode's and DigitalOcean's IP space. I deployed several mail exchanges and didn't have any luck reaching any addresses managed by AT&T's network. And, again, there's nobody I can kibbutz with to resolve it.
AOL has been such a hot tire fire that I ended up blackholing any outbound traffic to them. I know that sounds drastic, but anytime a single AOL user clicked the "spam" button on an email from one of my customers -- who weren't spamming, newslettering, or anything else remotely skeezy -- it would generate an automated complaint from AOL to Linode, and Linode would threaten to suspend my account. I'd have to dig the relevant traffic out of the logs and respond back to Linode with a polite "AOL's full of shit, please stop listening to them". I explained the situation to the few people that were impacted by it and everybody got on with their lives.
Microsoft's outlookprotection.com filter has been a gigantic pain recently too. It's annoyingly capricious and, again, the tools just aren't there to resolve it.
Email delivery has been a bit tricky for several years, but the last year especially it has become impossible for small services. You have SPF, DKIM, DMARC, great, but it turns out that Gmail also dings you if you don't have ipv6 records arranged right or if you aren't transporting mail over ssl or or or or.
Email was designed as a cooperative system but the BigCos have carved it up and are working hard to ensure that if a message doesn't come from one of their networks, then it's immediately suspicious.
Sendgrid isn't much better in this regard. My network received a flood of spam from them, with legitimate traffic mixed in. I couldn't block them without complaints and I couldn't not block them without complaints. I wondered if I was the only, so I signed up for their service and routed some mail through them for a few days to see what it was like as one of their subscribers. Turns out that Comcast and half a dozen other service providers hate them just as much and deliverability was around 79%.
The things you and others are experiencing, and that I experienced, are going to keep getting worse. The bigger networks are going to keep squeezing customers out of the smaller ones, breaking email bit by bit in the process. I hope Fastmail is able to grow quickly enough to keep sitting at the table.
The first RBL showed up around 1998, operated by Paul Vixie of DNS fame. I was using a small ISP in the East Bay at the time and they hated the MAPS RBL, especially the idea that they had to deal with some third party to resolve mail transport issues. But, from 1998 until the early 2000s, RBLs proliferated and lots of mail services subscribed to one or more. The one thing they had going for them was that they were easy to query, so it took no effort to find out if you were on the list, and if you were, there was usually a living, breathing human being somewhere that you could contact to get the matter resolved. It was a pain in the butt, but it could be handled.
During this time, the responsibility for ensuring that a message was received shifted from the recipient to the sender. Beleaguered systems administrators soon found themselves in a situation where they were supposed to be responsible for both ensuring that no spam reached their customers and that all of their customers' email reached the intended recipients.
Gmail came along and decided that, because they were operating "at scale", they didn't need to play in the same ecosystem. Over the years, ensuring that a message lands in a Gmail user's inbox has turned in to an infuriating game of trial-and-error. Gmail can do this because they now manage between 40% and 60% of the internet's email traffic.
AT&T/SBCGlobal/Yahoo/whoever they are now seem to have recently penalized all of Linode's and DigitalOcean's IP space. I deployed several mail exchanges and didn't have any luck reaching any addresses managed by AT&T's network. And, again, there's nobody I can kibbutz with to resolve it.
AOL has been such a hot tire fire that I ended up blackholing any outbound traffic to them. I know that sounds drastic, but anytime a single AOL user clicked the "spam" button on an email from one of my customers -- who weren't spamming, newslettering, or anything else remotely skeezy -- it would generate an automated complaint from AOL to Linode, and Linode would threaten to suspend my account. I'd have to dig the relevant traffic out of the logs and respond back to Linode with a polite "AOL's full of shit, please stop listening to them". I explained the situation to the few people that were impacted by it and everybody got on with their lives.
Microsoft's outlookprotection.com filter has been a gigantic pain recently too. It's annoyingly capricious and, again, the tools just aren't there to resolve it.
Email delivery has been a bit tricky for several years, but the last year especially it has become impossible for small services. You have SPF, DKIM, DMARC, great, but it turns out that Gmail also dings you if you don't have ipv6 records arranged right or if you aren't transporting mail over ssl or or or or.
Email was designed as a cooperative system but the BigCos have carved it up and are working hard to ensure that if a message doesn't come from one of their networks, then it's immediately suspicious.
Sendgrid isn't much better in this regard. My network received a flood of spam from them, with legitimate traffic mixed in. I couldn't block them without complaints and I couldn't not block them without complaints. I wondered if I was the only, so I signed up for their service and routed some mail through them for a few days to see what it was like as one of their subscribers. Turns out that Comcast and half a dozen other service providers hate them just as much and deliverability was around 79%.
The things you and others are experiencing, and that I experienced, are going to keep getting worse. The bigger networks are going to keep squeezing customers out of the smaller ones, breaking email bit by bit in the process. I hope Fastmail is able to grow quickly enough to keep sitting at the table.