Mailing Script

XML, Perl, Python, and other languages can be discussed here, even if it isn't PHP (We might forgive you).

Moderator: General Moderators

Post Reply
User avatar
shiznatix
DevNet Master
Posts: 2745
Joined: Tue Dec 28, 2004 5:57 pm
Location: Tallinn, Estonia
Contact:

Mailing Script

Post by shiznatix »

Here is the deal. I have been using Swift mailer to send out batch emails but it is getting to the point that the server times out and only part of them get sent. This is no good. I figure, php just isn't designed to do such a task of sending so many emails.

So what I was thinking was maybe I could compile a list of addresses, subjects, and emails and put them into a file or many files or whatever and just run a quick system() call that runs a python or something script that will do the busy work without taking up too many resources and whatnot. Anybody think this is an ok idea? If so, would python be the best idea for doing this? Are there maybe any pre-made scripts that would do this for me so I don't have to rely on php to do the hard work?
User avatar
ghurtado
Forum Contributor
Posts: 334
Joined: Wed Jul 23, 2008 12:19 pm

Re: Mailing Script

Post by ghurtado »

It's only a good idea if you are positive that PHP is the problem, and not the mail server you are using. Are you running your own mail server?
User avatar
shiznatix
DevNet Master
Posts: 2745
Joined: Tue Dec 28, 2004 5:57 pm
Location: Tallinn, Estonia
Contact:

Re: Mailing Script

Post by shiznatix »

The mail server is on the same dedicated machine. It does seam like the emails are sent, we get bounced emails and whatnot all day after sending a newsletter, but the script itself times out. I don't know how to see how many emails were sent from this script because the script times out, maybe because it is waiting for the mail server to finish but the mail server might be sending them out slowly as to not be too spammy. Since I can't see how many were sent I am just guessing that PHP is timing out and everything is sucking thus I wanted to use something that has no timeout time, like python or something.

Maybe there is something else wrong?
User avatar
onion2k
Jedi Mod
Posts: 5263
Joined: Tue Dec 21, 2004 5:03 pm
Location: usrlab.com

Re: Mailing Script

Post by onion2k »

You can't see what was sent? Doesn't the mail server have a log?
User avatar
shiznatix
DevNet Master
Posts: 2745
Joined: Tue Dec 28, 2004 5:57 pm
Location: Tallinn, Estonia
Contact:

Re: Mailing Script

Post by shiznatix »

onion2k wrote:You can't see what was sent? Doesn't the mail server have a log?
this is very noobish of me but I don't know what to even be looking for in the server logs let alone be able to count mails sent out. logs don't tell me anything. Either way though, i don't want the PHP script to time out like that, i want to know after I hit "submit" how many emails will be sent out, which is doable through the python way i was thinking. Is anyone else using the swift batchSend() to send out around 10,000 emails at a time?
User avatar
EverLearning
Forum Contributor
Posts: 282
Joined: Sat Feb 23, 2008 3:49 am
Location: Niš, Serbia

Re: Mailing Script

Post by EverLearning »

Better idea would be to send emails in batches of about 100, 200 emails at a time, per script execution, until all are sent. That way your script won't timeout and your mail server won't be overloaded. If you're using cron you can set your script to run every minute and to see if there are mails left to send.
User avatar
ghurtado
Forum Contributor
Posts: 334
Joined: Wed Jul 23, 2008 12:19 pm

Re: Mailing Script

Post by ghurtado »

Also, if you need to ensure that all emails are sent, I would recommend logging from your script either to a text file or (preferably) a database, so that even if the script is interrupted (python scripts can finish prematurely too... :) ) you will know exactly the work that was done before it aborted.
User avatar
EverLearning
Forum Contributor
Posts: 282
Joined: Sat Feb 23, 2008 3:49 am
Location: Niš, Serbia

Re: Mailing Script

Post by EverLearning »

Here's how I typically handle sending large mail batches in my applications(using Swift Mailer)
  • 1) When admin clicks on "send newsletter" in admin section, script selects all mails that need to be sent and inserts them into a table "newsletter_recipients", which has fields: email and retries. I choose approach with inserting, because I collect emails from several tables in the DB, and didn't want to litter tables with additional fields whose only purpose is to reflect the sent email status.
  • 2)Cron script, which runs every minute, checks if there is anything to send. At the beginning of it, there is a check to see if the script is already running, so we don't have multiple copies of the script sending mails at once, overloading the mail server. If there are mails in the table, first 100 mails which have retries field with value less than 3(for example, you can use whatever number suits your purposes)are selected.
  • 3)Mails are then sent one by one, using $swift->send() instead of $swift->batchSend(), for greater control. If the sending is successful, email is deleted from the table, if not, retries value is incremented by 1.
  • 4)When all the mails are sent, left in the table are mails whose sending had been retried a couple of times and failed, so you can add that to some kind of report. This will just have mails that your mail server didn't accept. If you want to know what mails weren't delivered successfully you'll have to set up a return-path mail address, and then read the reports for failed deliveries from there.
Last edited by EverLearning on Thu Aug 21, 2008 3:17 am, edited 1 time in total.
alex.barylski
DevNet Evangelist
Posts: 6267
Joined: Tue Dec 21, 2004 5:00 pm
Location: Winnipeg

Re: Mailing Script

Post by alex.barylski »

If PHP is timing out can't you just up the execution time?

What MTA are you using? Are you sending by connecting via SMTP?

If your emails are bouncing that isn't the fault of PHP.
User avatar
shiznatix
DevNet Master
Posts: 2745
Joined: Tue Dec 28, 2004 5:57 pm
Location: Tallinn, Estonia
Contact:

Re: Mailing Script

Post by shiznatix »

Hockey wrote:If PHP is timing out can't you just up the execution time?

What MTA are you using? Are you sending by connecting via SMTP?

If your emails are bouncing that isn't the fault of PHP.
Upping the execution time to like 30 minutes is a bit too much for me. I am sending via SMTP. I know the bounced emails are not PHPs fault, I am just wondering if all emails are being attempted or not.

EverLearning: sounds like a good idea. I will look into that and probably do it that way. Just seamed fancy-funtimes(R) to use a python script to do it.
Post Reply