Using G Suite to receive kenyonhill.com emails is starting to work now.
Sending emails is still not working and I'm unsure why. Also, I found they have limits for relay users like me that are lower than our current usage. I think they are limiting it to 130 emails sent per day (according to https://support.google.com/a/answer/2956491#sendin...
we have 1 user license [which are $5/mo each] ) A quick check of the old server showed that 672 emails were sent out yesterday and that seemed typical.
I could cut back on outgoing emails to try to get under the limit. It looks like a lot go out in the morning (~250), probably abandoned account reminders (could just stop doing that). Also, I suspect immediate pm notifications might be generating a lot of emails. It's not something I could fix fast, but over time I could probably cut it down to a smaller number.
Another provider, Sendgrid has a relay service that lets you send 40k emails/mo for $10/mo. So, that may be an alternative, maybe just temporarily until I can get out number down. Receiving emails would still be an issue, but G Suite can probably handle that (for $5/mo).
It might be a few days before I can it working again. I wasn't prepared for this. At least the website seems to be working well.
I'm still working on this email issue. Emails are going out now and seem to work in most cases. Yesterday, the emails had some tracking gunk in them (e.g. the links I put in them were replaced with wrappers that track if you clicked or not). Also, there was an invisible pixel in them that tracked if you opened the email. SendGrid did this by default but I've turned it off this morning.
I still need to do a lot for sorting out incoming emails, but that's mostly an internal thing for me (won't affect people).
Probably the toughest problem is that email notifications from gametz to users that have Microsoft-based emails (e.g. hotmail.cim, live.com, or outlook.com ), are being blocked. I found a form to submit to them asking them to let me emails go through, so I did that. But, it's hard to tell if it will work and I don't have much control over what Microsoft does. So, that sucks. But, with a little luck, they may unblock me. Searching for the issue, you can find tons of people complaining about it in forums. I think I had the issue in the past too and the form worked. We'll see.
Maybe, but it's probably more complicated than that. Email is just this convoluted thing these days with various systems in place to try to block spammers.
It's not that uncommon for a domain to use another to send email for them. When I used G Suite, it was google's mail servers that also don't match my domain. Given that many hosting providers are blocking mail ports in some way, many sites are going to be in this position. There are companies that provide mail services to help them, so it's also a business opportunity.
To get G Suite and SendGrid to take over GameTZ's email, I have to put a MX record into my DNS that tells everyone that they are my mail server. So, that's pretty official and it could generally only be done by the domain owner. So, it's not a case of hijacking a domain to spam. I don't think domains not matching the best indicators that someone is spamming. There is also some trick that SendGrid had me do that I think causes it to look at bit like it really is my domain too (some odd CNAME records).
It seems pretty clear to me that SendGrid is used primarily by "email marketers" as their UI caters to that sort of thing. So, it also seems likely that emails from their servers get reported as spam (because they are, in a sense, to some people [including myself]). So, I suspect it's as simple as Microsoft getting lots of reports of spam from SendGrid's servers, so they block their server's IPs. Thus, I am affected. But, perhaps telling them I'm legit will help them separate my emails... hoping. It is Draconian of them to block all from SendGrid a surely there are plenty of legit emails coming from those servers too. And, I assume even Microsoft is OK with email marketing.
In other news, it appears there may also be an issue with yahoo@com emails. I'm getting multiple deferrals from them that don't seem to be resolving. Maybe there's some form I can fill out for them as well..
gmail emails are working fine at least. Also, most uncommon email providers are working.
I got a reply back from yahoo, so I provided them with more info, but their last reply was essentially boiler plate brush off. So, that's not looking good. I got a reply back from Microsoft that told me to resubmit correctly. So, I'll try that again. Part of the issue with both of them is that they want me to give them info that only a mail server has. I'm no longer a mail server, so I can't even answer some of their questions. SendGrid is my mail server. I contacted their support to try to get the info from them. Their answer was also boiler plate and suggested I upgrade my plan with them (as noted before this would be from $10/mo to $90/mo [more than it costs to host the server with google cloud]). Even with the upgrade, I'm unsure it would fix the problem. But, $90 is just way too much in my opinion.
Today, I'm trying to drill down more into Google Cloud to see if there's some way to get the ports unblocked. Yesterday, I sent a very whiny suggestion to them about the situation, assuming they would not even consider it. This morning, after looking at their support forums, I found that the blocks of email ports is a top issue for them (based on customer votes). And, I think I'm seeing some sign that they do open ports if you can make a case for being legitimate. I am legit, so I'm going to do my best to try to get that. Though, contacting Google Cloud support does not seem easy or even possible from what I can tell so far. I may need to pay more for support? wtf
Meanwhile, emails to hotmail, live, outlook, aol, and yahoo are not working. So, for example, if someone has an email with one of these and they try to make a new account, they will be unable to get access to the site because you have to verify your email to get a password and login. I just got an email this morning from a guy with a hotmail email address saying that he hasn't got his password and doesn't know what's wrong. I can't even reply to him because my reply will get blocked too. I was going to email him from my private gmail account just to let him know, but looking him up he actually had past account with bans for piracy and BTRs. So, yeah... whatever, I'll just let that one go.
Existing users with those blocked emails aren't getting notifications (maybe not so bad but may cause people to be slow to respond to offers/trade we'll see).
I've actually done a fair bit of work to integrate with SendGrid. They have an API that is quite good, a nice solution to the problem of blocked ports. So, I'm able to use that effectively. But, their mail servers being blocked undermines this. Clearly, the business model is a sort of bait-and-switch where the promise email services for free or $10/mo, but ultimately, if you want it to actually work, you need to pay at least $90/mo.
If we're stuck with SendGrid (hey it works with gmail users and many others), I could try to notify gtz users with hotmail, yahoo, aol, etc,. emails to let them know the situation at least. Many can likely get by without email notifications. For new accounts, I could just not allow email addresses from them to be used. Maybe, I could get rid of the email verification step for new accounts. Though, that would likely trigger more forum spammers to come here and that's a pain in the butt to deal with. I don't love this situation (e.g. trying to workaround the blocked emails), but may have to accept it.
I could try different hosting, but it will cost a bit more than Google from what I can tell and I think I'm seeing similar restrictions on emails with most of them. It's hard to tell what the limits are and how they would affect what GTZ does until I set it all up. Setting it up is a ton of work. So, I'm disinclined to do this unless I'm more sure. Google Cloud seems really good otherwise too, so I'd like to stay with them.
Hopefully, I can get through to Google Cloud Support and get my email ports opened.
If it helps with lowering emails sending out i can turn email option off if theres any
Because i have app giving me notifications for almost everything
I don't think so. There were some DNS propagation waits initially to get my MX records hooked up with them, but that was under an hour. I've have my TTLs turned way down so DNS records expire in 5 minutes.
From the support emails I exchanged with yahoo, microsoft and sendgrid; the issue was generally agreed by all to be that yahoo & microsoft were blocking sendgrid's IPs because they had been used to send spam. Though, it was never quite stated clearly (instead they all danced around this, but implied it at the same time). One email said they had been unblocked once before but that spam continued, so they reblocked it and weren't planning to do that again.
SendGrid did say they were working hard to get their IPs unblocked. I urged yahoo/microsoft to make an exception for me, but they kept replying obtusely that I should following their tips on how to look less like a spammer.
My best guess at this point is that SendGrid has a number of IPs in use by their mail servers. For the shared IPs that I'm using, some are blocked, some are not. Some are dedicated and cost a lot and likely aren't blocked. Possibly, it's just random which SendGrid server sends my mail and thus I might get lucky and get a non-blocked one sometimes. Or, SendGrid may be selecting the server in some way and may move me to better IPs over time if they think I'm not a spammer. I think that's too much to expect as I'm also not a paying customer yet and I'm not likely to ever pay them much.
Checking again just not it seems that hotmail is blocking again (sometimes, not always, seems to flip back and forth within the last few hours). Yahoo is working more consistently, though. I do see a different IP on the working hotmails than the blocking ones, though it's hard to be too scientific (takes multiple click to drill down in their UI for each email to get the IP of the server.
I'm trying not to care so much. For a week (since I moved the server), about 40% of emails going out were not delivered, but I got very few complaints about it. It has bothered me a lot, but maybe I'm overreacting. I'm not sure users care that much and even if they do, maybe they can deal with it, work around it, whatever. The site is honestly in decline and maybe part of that is things will not work as well as we go forward. It's not for lack of trying on my part, but I'm running into a lot of brick walls. I can keep trying, but maybe I'll just throw in the towel instead of doing a ton more work (like moving to AWS).
I'm still hoping Google will open my ports. If they do, I think we're fine (just like it was before -- though, I'll have to do some work to set that back up and I think they'll want me to upgrade a few things too). If they say "no", we'll see. Maybe they'll give me some recommendation otherwise that will help me. Or, I could work hard to minimize emails and emphasize alternatives.
Someone just suggested I try to setup a Web Push API thing. That could help replace emails notifications for many people. I'd need to look into it more, but I think it's like app notifications except for browsers (thus, more universal perhaps). It still doesn't seem like it would replace email verification. But, perhaps emails could only be used for that and almost no other notifications. It would be a lot more work on my part. I'm not sure I'm up for it or that it makes sense given the site's decline. But, I like to program stuff, so maybe.
> I'm trying not to care so much. For a week (since I moved the server), about 40%
> of emails going out were not delivered, but I got very few complaints about it.
Thanks for all the insight on everything. Interesting, and a little depressing to hear again that the site is sliding slowly downhill.
I think the above line though might be a problem for you eventually, though. I'm guessing that you're getting few complaints because people haven't noticed that they didn't get their mail. How would they know if a mail wasn't delivered that should have been? Nothing to complain about it if you can't figure out there's a problem.
Yes, they might come to the site and see pending PM's or trade messages, but they might not. Without the e-mail, you're missing that vital notification step, and I wonder if sooner or later someone's going to miss a trade message that leads to a bad situation (BTR).
Agreed. I may try to put in some sort of metrics to get counts for what type of emails are being sent out and why. But, from looking at the SendGrid activity list, most of it is trade related (e.g. subjects like, "New offer from.." "Offer Message Added", "Offer Declined", "Offer Agreed Upon"). Those are clearly for important core features of the site, but at the same time, may not be essential or even being read by the people getting them.
It's hard to say, but initially sendgrid was putting in tracking (like an invisible image load when an email is read and a wrapper around clicks) and the percents were fairly low. I suspect some people just turn on all notifications, then ignore them or don't check their email or are just on the site so it's wasted.
Still, I'm sure others rely on it, so I can't make assumptions. It will be hard to sort out. Which is why, I'm still hoping to use my own mail server again.
> Agreed. I may try to put in some sort of metrics to get counts for what type of
> emails are being sent out and why. But, from looking at the SendGrid activity list,
> most of it is trade related (e.g. subjects like, "New offer from.." "Offer Message
> Added", "Offer Declined", "Offer Agreed Upon"). Those are clearly for important
> core features of the site, but at the same time, may not be essential or even being
> read by the people getting them.
> It's hard to say, but initially sendgrid was putting in tracking (like an invisible
> image load when an email is read and a wrapper around clicks) and the percents were
> fairly low. I suspect some people just turn on all notifications, then ignore them
> or don't check their email or are just on the site so it's wasted.
> Still, I'm sure others rely on it, so I can't make assumptions. It will be hard
> to sort out. Which is why, I'm still hoping to use my own mail server again.
You could probably only send the "New offer from" emails and eliminate the rest. Even if I don't actually read the email I still see the "New offer from" subject and know to come check the site. I also like to get emails for a PM. I think those are the only 2 important emails for me.
I checked SendGrid this morning and it says this at the top: "Your account has been suspended. Please contact Support to regain access. [Contact Support]" The support page about it seems to imply that I'd get an email about it, but I haven't yet. It also says, "In most cases, SendGrid Support cannot reactivate a warned, suspended, deactivated or banned account, so contacting them directly may delay reactivation." So, I guess I'll wait to see if they contact me. Some emails still seem to be work (gmail at least), others not so much.
Maybe this is just timing as most of the emails sent after 6am are about "New Matches". Maybe a bunch went out all at once (normal for GTZ mornings, so this probably happens every day) and SendGrid sees that as suspicious perhaps.
*shrug* I could try to slow them down or something or not send them or send them by alternate notification method. Matches aren't generally seen as so great by most traders anyway, as far as I know.
SendGrid replied to my support request saying they had detected suspicious activity from me and thus suspended. They gave me a lot of questions to answer (basically asking if I'm a spammer in various ways). I have now answered them explaining how I'm not a spammer at all.
No emails were getting through most of yesterday, so I stopped sending emails. GameTZ is currently not sending any emails for anything.
If SendGrid unsuspends me, I'll probably delete the emails waiting (most are probably stale and if they all go out at once, it may look suspicious again), then restart emails.
I'm still hoping for Google to unblock my mail ports... maybe Monday? If that happens, I can stop using SendGrid and hopefully go back to emailing the same way we used to.
If Google says, "no" ... I'm not sure what I'll do. Perhaps reduce emailing in general, encourage alternate ways to notify members and stay with SendGrid despite inconsistent delivery. Or, move the server to AWS and hope they let me run my own mail server. Maybe I can somehow partially setup the AWS server then make the request to open the mail server first, then move if they say "yes".
If Google continues to stall, I may still consider some of the above "solutions" as it has been a long time since I initially asked them to unblock me and the longer I wait, the less likely it seems they'll help.
I decided to bite the bullet and disable most of the email notifications, at least for now. In some cases, if it made sense, I replaced them with system pms. The emails I kept were for account creation, suspension, password reset, moderator emails, payments, and rating requests (tricky as I need a reply... might try to do this as a pm from me, but that has issues too... )
But, pm/offer/trades not longer notify by email, which I understand is a loss. Not that any emails is going out now, but assuming it gets unsuspended, I'm trying to focus on the bare minimum. I think this could help in other situations too (looking at AWS, they have limits as well).
I can also try to work on web API notifications as yet another way to get notified...
Google said no to port 25 but yes to the other ports. I wasn't expecting that. And, it's actually no good. I'm not an expert in mail config, but if I can't use port 25 outgoing from my server, I can't send mail (mostly, as far as I know). I replied to them, trying to explain it again and expressing my frustration. Maybe I'm missing something, we'll see.
So, I don't know. SendGrid is still kind of working. And, I am sending out less emails now. I could stay with them. I think it would be free and I like their API for other reasons. Though, emails definitely still get blocked sometimes too (bad).
I could move the who site to AWS and go through the process with them, hoping they unblock my ports. Crap shoot basically.
I'll keep trying to talk to Google about it too. Maybe it is possible to do mail without 25? Or maybe I can talk them into it (doesn't seem likely). Honestly, waiting 8 days for this reply had lowered my opinion on them. They don't seem to really understand my problem. I wonder if they read what I wrote even.
One important thing to keep in mind is I haven't had to pay either Google Cloud or SendGrid, so yeah, hard to complain exactly.
I can try to post in a google cloud forum about it too...
SendGrid has closed the ticket and that suspension seems resolved/over. Maybe I've solid with them from now on. A quick check just now showed most emails being delivered as well (including hotmail and yahoo). If SendGrid worked like this all the time, I could just stay with them and be fine. Or, if the percent working was like 90%, I could live with that. It had been as low as 60% which didn't seem good enough.
Google Support just said, "I understand the importance of using the port 25 within your project. I'll provide your reply to our Compute Engine team to request opening the port 25 within your project according to your scenario. Keep in mind that is not guaranteed open it due security reasons. I'll give you an update on Friday or as soon as I have a reply from our Compute Engine team."
So, maybe there's still a chance they'll unblock me. Just an hour before it seemed fairly hopeless.
Looking at AWS again, they have something called SES which seems like an in-house SendGrid-like service. The limits are very high to use it for free, so I could that. But, they also offer dedicated IP for $25/mo. So, that seems to imply the shared IPs get blocked (like SendGrid). But, $25/mo for a dedicated IP is the cheapest price I've seen too (still more than I'd prefer to pay, but could just do it). Moving to AWS would be some work, but mostly just repeating what I already did for Google Cloud (twice, since I did it as a test first). And, I'd get to say I used AWS EC2 (fun!?). Also, using Google Cloud is a little annoying because it keeps making me switch my google account between personal/business (in Chrome/gmail/sheets/etc -- I use a ton of google stuff for personal things)...
Anyway... thanks for reading this (not even sure why I keep posting about it, but I guess it helps to talk about it or something).
Google just replied saying they will open all my mail ports, including 25 (" within 4 days starting on the next Monday" ...wat?) Anyway,
and hopefully this means we can go back to how things were... in a week or so. Though, I shouldn't count my chickens given all the bad luck I've had with this. I'll believe it when I see it, but this is potentially great news.
I will likely keep some of the emails off and scale back somewhat regardless as it did seem like many of the emails were kind of random and likely not appreciated (e.g. reminders and such). And, that may help me avoid being reported as spammer in the future.
Also, I'd still like to see if I can get the web push api thing going as yet another notification alternative for people to use.
> Google just replied saying they will open all my mail ports, including 25 (" within
> 4 days starting on the next Monday" ...wat?)
lol, depending on the person, "this Monday" would be the 14th, making "next Monday" the 21st. Which will it be??
PM (immediate) email notifications are also tricky. There can be lots of them and can be of low value (redundant if you're on the site or just unimportant stuff). But, some pms are really important too and getting notified when they happen is great.
I think for stuff like that it may be better to rely on app and/or web notifications as they are more ephemeral and will clear themselves in some cases (e.g. if you read the pm, the notification thingy or badge number goes away).
Someone had suggested I turn off email notifications for everyone, then let them turn them back on if they still want them. That seemed like a decent idea as it would clean out all the non-serious users.
There's also the daily/morning New Replies and New Matches emails. Not sure about those. I noticed a lot go out one day (the day SendGrid suspended me), so that gave me pause. I know someone said they love getting their New Replies email every morning.
For some stuff, I migrated it to a "system pm". For example, if a mod moved your topic, it will now just pm you about it instead of emailing. I'll probably keep that like that. Maybe there are other cases where I can replace an email with a pm like that. System pms are odd as you can't reply and are not with another user, just a single message.
I'd like to rethink/rework notifications, in general. But, it's also complicated with many ways to do them and many personal preferences.
If I'm able to add Web Push notifications into the mix, I'm if the existing on-site pop-ups will still make sense.
I may make a new topic about it next week or whenever I have my mail server back. I'd definitely like to get more feedback about it and explain it more to people (I'm sure this topic has been hard to follow).
I saw some mails ended up in spam I didn't notice till now I marked them not spam
> Someone had suggested I turn off email notifications for everyone, then let them
> turn them back on if they still want them. That seemed like a decent idea as it
> would clean out all the non-serious users.image
Google responded to the support ticket yesterday saying something had gone wrong and they'd try again.. that I should give it until Friday. I've lost faith that anything will happen, but I'm tired and just want to give up. I think SendGrid may be fine, so will just do that assuming Google is just blowing hot air. I'm still paying for Google silver support (to access this so-far useless ticket). I'll cancel that on Friday either way.
I just enabled offer-related emails today. So, emails will go out (if enabled in settings) for offer updates like agreed upon, declined, new message, etc. Previously, it was just for new offers (which I'd prioritized).
After I enabled the trade-related emails, the daily emails sent by GameTZ jumped from ~50 to ~200. We'll see how much this increases that number.
Next, if all goes well, I'll try restoring pm-related emails. I'm unsure about the rest.
> I still can't get over how people still use yahoo or aol for anything.
Benefits packages that are offered through aol is why I am still with them. I would be paying more money elsewhere otherwise for those benefits.
Tech Support on by computers and laptop, anti virus, and identity thief. The tech support is pretty useful cause if there is anything wrong they look it over. I remember having like 3 issues that took a long time to correct. Best Buy offers that tech support service for $100 a year but its included in as an aol member.
Yeah thats why i switched to gmail, yahoo/aol mails are dead anyway since they got rid of IMs
> People that still use yahoo and aol should just switch to something that doesn't
> suck, tbh.
> Changing email is a lot to ask of someone. It's not something I'd expect a user
> to do.
Right, but they should still want to do it. I get that it's a pain, because I've done it but AOL, Yahoo, hotmail etc... are actively terrible.
> Remember that people said the same thing in the past though. When people were using
> AOL, others went to Yahoo or Hotmail and said that AOL people were outdated and should
> switch away from an email where you are tied to a paid provider. Now, people are
> saying that Yahoo and Hotmail are outdated/terrible/whatever and that they should
> use something like GMail. One day, we'll probably be saying the same thing about
> the GMail users, of course -- because they didn't switch to MagicHyper3DMail.
You are likely absolutely right. Technology changes, and improves. I've switched my main email twice.