Delayed Email Notifications

14 posts / 0 new
Last post
#1 9 April, 2014 - 10:20
Offline
Joined: 2 years 1 week ago

Delayed Email Notifications


Is anyone noticing delayed email notifications? I'm not sure if it's just me.

It may be that the email notification queue is getting so long that the server can't process them fast enough and I will have to make some adjustments.

9 April, 2014 - 14:38
Offline
Joined: 3 years 4 months ago

Yes, I've noticed for the past couple weeks, I always get the emails at one time. I don't get them when a new comment is on a post, they just all come at once. I have it set on "immediately" though.

I actually haven't received any in a week or so, I'm not sure whether it's because I read everything within a couple hours, or I might have somehow unsubscribed to everything. After checking, I still have 198 active subscriptions, so I'm not sure what that is.

I don't actually check my e-mail for responses to posts and stuff, I just check the "unread" tab. But I have noticed delayed messages that come in bulk.

Bateman

Edit: Looking at my e-mail address, I last received messages from this site March 31st, before that March 24th, and March 18th. Seems suspiciously close to a once a week schedule, even though I'm pretty sure I've always had it on a "immediately" schedule. Also haven't received any April 6th, so it doesn't seem to fit...

9 April, 2014 - 15:19
Offline
Joined: 2 years 1 week ago

Thanks... that helps answer my question.

Originally, the site would send the email notifications at the same time someone posted a new comment. On really long threads, this would cause the server to slow down which would encourage people to click the submit button again. I had a suspicion that that was causing the double posts, so I changed the settings so that email notifications are put in a queue and sent once per hour.

I think what is happening is that the email queue tends to grow much larger than can be processed in that narrow window of time once per hour.

I will see if I can set up a cron job to run the email delivery every 15 minutes or so, but there is a delicate balance between processing the emails and overwhelming the server. :)

9 April, 2014 - 15:23
Offline
Joined: 2 years 1 week ago

A quick check of the logs makes this seem likely.

"Reached processing limit on queue processing:"

9 April, 2014 - 15:50
Offline
Joined: 2 years 1 week ago

I just found the settings, and there were 628 queued emails. It only had four seconds per hour to work on the queue, so was falling behind. I'm trying a couple of things now that I hope will fix it...

9 April, 2014 - 16:00
Offline
Joined: 2 years 1 week ago

My apologies to anyone who just got a lot of email from the site.

I'm hoping that it's fixed now. :)

9 April, 2014 - 16:06
Offline
Joined: 3 years 4 months ago

Glad it's fixed. Is it going to send 4 times an hour now? Every 15 minutes?

Why did it only have 4 seconds an hour?

mn.png

9 April, 2014 - 22:14
Offline
Joined: 2 years 1 week ago

Sorry about that email overload. :/

Every hour, the website runs a process called "cron" which performs maintenance tasks like sending the notification emails. Whichever user hits the site during that process the triggered the cron gets a delay. To reduce the length of that delay, I limited the settings to four seconds for sending the emails.

I fixed things so that the site now runs cron independently of user visits and spends eight seconds to send the emails.

Since this afternoon, the email queue is still at zero, so it looks like it's now keeping up with the number of new posts. I'll keep an eye on it over the next few days.

Currently, it should send about every 30 minutes, just to keep load off the server, but if that isn't fast enough, I'll try increasing it and see if the server can still handle it without going offline.

10 April, 2014 - 08:44
Offline
Joined: 2 years 12 months ago

Thanks, I was wondering about that. At first I thought I must be saying nothing that anybody was interested in, then all of a sudden everybody was caring.

10 April, 2014 - 17:27
Offline
Joined: 2 years 1 week ago

Sorry about that. The current settings (15 minutes now) are keeping the queue cleared, so the problem appears to be fixed for now.

14 April, 2014 - 10:05
Offline
Joined: 2 years 1 week ago

When I woke up this morning, the queue was backed up again with 162 emails. I think that the site can handle new threads, but when people post to the longer threads, the site prepares a couple hundred emails to send out.

I'm processing the emails this morning. If anyone gets too many at once, let me know and I'll see what else I can do to fix it.

23 April, 2014 - 17:36
Offline
Joined: 4 years 8 months ago

Are the default settings set to subscribe people to a thread? Perhaps changing the default will mean only those who truly want a notification have to worry about it. Just a thought anyway :) Could always offload email processing to a separate server as well

24 April, 2014 - 12:36
Offline
Joined: 2 years 1 week ago

People choose the settings when they sign up, but I think it defaults to yes. I will double check that.

Moving the email notifications to a separate server is something that I would like to do.

It looks like the current settings are working though. If anyone notices too much email, just let me know and I will try to optimize it a bit more. :)

13 July, 2014 - 11:08
Offline
Joined: 2 years 1 week ago

This problem may have just temporarily returned due to the server change. I forgot to setup the cron job on the new server, so the queue got backed up again.

Sorry in advance if anyone just received a flood of emails. :/

Learn memory techniques for free! Just click the "Sign up" button below to create an account and we'll send you an email with some tips on how to get started.

Related content: