300+ e-mails per hour with different content
Moderators: Chris Corbyn, General Moderators
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
300+ e-mails per hour with different content
I run a website which sends out over 300 e-mails per hour each with different user-specific content.
What is the best method of doing this?
What is the best method of doing this?
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Depending upon the amount of variation between these emails you may be interested in looking at the decorator plugin. 300 emails per hour is quite low, but in the interest of scalability I'd probably do it by cron. You throw the emails into a database, marked as "unsent", then the cron script runs every minute or 2 sending all those "unsent" and marking them as sent.
EDIT | Forgot the link:
http://www.swiftmailer.org/wikidocs/v3/ ... /decorator
EDIT | Forgot the link:
http://www.swiftmailer.org/wikidocs/v3/ ... /decorator
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
- Each e-mail will be totally different to the one before itd11wtq wrote:Depending upon the amount of variation between these emails you may be interested in looking at the decorator plugin. 300 emails per hour is quite low, but in the interest of scalability I'd probably do it by cron. You throw the emails into a database, marked as "unsent", then the cron script runs every minute or 2 sending all those "unsent" and marking them as sent.
EDIT | Forgot the link:
http://www.swiftmailer.org/wikidocs/v3/ ... /decorator
- 300 will go up so it does need to be scalable
- I currently have a cron job run every hour
- The cron runs a script which grabs their e-mail address from a database, creates the messages, then sends the e-mail
I am guessing, from what you have said that I am going about the whole process backwards?
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
You sound as though you're going about it in a sensible way actually
I assume you don't just pull out however many emails happen to be there and send them all at once? I'd probably run the script more than once and hour (it wouldn't hurt to run every few minutes if there's no mail to send most of the time). I usually suggest keeping batch sizes to no more than 100, although there are ways around it using the AntiFlood plugin.
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
There has to be a strict hour between each e-mail sent.
So, User1 should get one e-mail every 60 minutes! The same goes for all 300+ users.
The script I use simply queries the database, finds the required people and within a "while" statement sends the e-mails. So in answer to your question, it does try and send them all at once (stupid, I know!).
I know I should look at batch processing but the way in which the message is created and users grabbed from the database is already complex enough!
So, User1 should get one e-mail every 60 minutes! The same goes for all 300+ users.
The script I use simply queries the database, finds the required people and within a "while" statement sends the e-mails. So in answer to your question, it does try and send them all at once (stupid, I know!).
I know I should look at batch processing but the way in which the message is created and users grabbed from the database is already complex enough!
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
I see, I assumed this was just a newsletter or something. Is this some hourly report? Use the AntiFlood plugin to deal with that level of batching then.CRichardson wrote:There has to be a strict hour between each e-mail sent.
So, User1 should get one e-mail every 60 minutes! The same goes for all 300+ users.
The script I use simply queries the database, finds the required people and within a "while" statement sends the e-mails. So in answer to your question, it does try and send them all at once (stupid, I know!).
I know I should look at batch processing but the way in which the message is created and users grabbed from the database is already complex enough!
http://www.swiftmailer.org/wikidocs/v3/ ... /antiflood
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
Yea, you could call it that. The content will most likely change every hour! The same number of e-mails per hour will be sent (to a degree).
I've tried the AntiFlood plugin, which works up until 101 e-mails.
I've had so many problems, including:
- mysql connection closing itself
- php timing out
- memory/bandwidth usage
- smtp usage
I've tried to log everything, still no real result!
I've tried the AntiFlood plugin, which works up until 101 e-mails.
I've had so many problems, including:
- mysql connection closing itself
- php timing out
- memory/bandwidth usage
- smtp usage
I've tried to log everything, still no real result!
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
Note: when I run the script in FireFox after a certain amount of time it wants me to download the file and in IE after the same amount of time it gives me "Page cannot be displayed".
The bandwidth monitor gave me the following going out:
2625
34660
53480
56080
62688
137839
143497
146122
149734
That's just 9 e-mails! I logged the monitors results to a database.
The bandwidth monitor gave me the following going out:
2625
34660
53480
56080
62688
137839
143497
146122
149734
That's just 9 e-mails! I logged the monitors results to a database.
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
-
CRichardson
- Forum Newbie
- Posts: 12
- Joined: Sat Jun 02, 2007 7:22 pm
What is the best way to ensure you don't create an infinite loop?
Is that infinite? Surely not because it only runs till there is no more rows left?
Code: Select all
while($fetch=mysql_fetch_assoc($result)) {
}- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
FYI, readings are in bytes so this is quite low really. The bandiwdth monitor is logging the total bytes sent, not the bytes for each individual email, this is why it goes up as the number of emails goes up.CRichardson wrote:The bandwidth monitor gave me the following going out:
2625
34660
53480
56080
62688
137839
143497
146122
149734
That's just 9 e-mails! I logged the monitors results to a database.