300+ e-mails per hour with different content

Swift Mailer is a fantastic library for sending email with php. Discuss this library or ask any questions about it here.

Moderators: Chris Corbyn, General Moderators

Post Reply
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

300+ e-mails per hour with different content

Post by CRichardson »

I run a website which sends out over 300 e-mails per hour each with different user-specific content.

What is the best method of doing this?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Depending upon the amount of variation between these emails you may be interested in looking at the decorator plugin. 300 emails per hour is quite low, but in the interest of scalability I'd probably do it by cron. You throw the emails into a database, marked as "unsent", then the cron script runs every minute or 2 sending all those "unsent" and marking them as sent.

EDIT | Forgot the link:

http://www.swiftmailer.org/wikidocs/v3/ ... /decorator
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

d11wtq wrote:Depending upon the amount of variation between these emails you may be interested in looking at the decorator plugin. 300 emails per hour is quite low, but in the interest of scalability I'd probably do it by cron. You throw the emails into a database, marked as "unsent", then the cron script runs every minute or 2 sending all those "unsent" and marking them as sent.

EDIT | Forgot the link:

http://www.swiftmailer.org/wikidocs/v3/ ... /decorator
- Each e-mail will be totally different to the one before it
- 300 will go up so it does need to be scalable
- I currently have a cron job run every hour
- The cron runs a script which grabs their e-mail address from a database, creates the messages, then sends the e-mail

I am guessing, from what you have said that I am going about the whole process backwards?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

You sound as though you're going about it in a sensible way actually :) I assume you don't just pull out however many emails happen to be there and send them all at once? I'd probably run the script more than once and hour (it wouldn't hurt to run every few minutes if there's no mail to send most of the time). I usually suggest keeping batch sizes to no more than 100, although there are ways around it using the AntiFlood plugin.
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

There has to be a strict hour between each e-mail sent.
So, User1 should get one e-mail every 60 minutes! The same goes for all 300+ users.

The script I use simply queries the database, finds the required people and within a "while" statement sends the e-mails. So in answer to your question, it does try and send them all at once (stupid, I know!).

I know I should look at batch processing but the way in which the message is created and users grabbed from the database is already complex enough!
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

CRichardson wrote:There has to be a strict hour between each e-mail sent.
So, User1 should get one e-mail every 60 minutes! The same goes for all 300+ users.

The script I use simply queries the database, finds the required people and within a "while" statement sends the e-mails. So in answer to your question, it does try and send them all at once (stupid, I know!).

I know I should look at batch processing but the way in which the message is created and users grabbed from the database is already complex enough!
I see, I assumed this was just a newsletter or something. Is this some hourly report? Use the AntiFlood plugin to deal with that level of batching then.

http://www.swiftmailer.org/wikidocs/v3/ ... /antiflood
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

Yea, you could call it that. The content will most likely change every hour! The same number of e-mails per hour will be sent (to a degree).

I've tried the AntiFlood plugin, which works up until 101 e-mails.

I've had so many problems, including:

- mysql connection closing itself
- php timing out
- memory/bandwidth usage
- smtp usage

I've tried to log everything, still no real result!
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

Note: when I run the script in FireFox after a certain amount of time it wants me to download the file and in IE after the same amount of time it gives me "Page cannot be displayed".

The bandwidth monitor gave me the following going out:
2625
34660
53480
56080
62688
137839
143497
146122
149734

That's just 9 e-mails! I logged the monitors results to a database.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Wanting to download the page is usually a sign of a broken PHP installation. Infinite loops or infinite recursion sometimes cause this behaviour too. You'll need to post the code you're using if I'm to be able to assess that could be causing that.

As for timeouts, you need to set_time_limit().
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

What is the best way to ensure you don't create an infinite loop?

Code: Select all

while($fetch=mysql_fetch_assoc($result)) {

}
Is that infinite? Surely not because it only runs till there is no more rows left?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

CRichardson wrote:The bandwidth monitor gave me the following going out:
2625
34660
53480
56080
62688
137839
143497
146122
149734

That's just 9 e-mails! I logged the monitors results to a database.
FYI, readings are in bytes so this is quite low really. The bandiwdth monitor is logging the total bytes sent, not the bytes for each individual email, this is why it goes up as the number of emails goes up.
Post Reply