Page 1 of 1

Mailing Script

Posted: Fri Aug 15, 2008 8:23 am
by shiznatix
Here is the deal. I have been using Swift mailer to send out batch emails but it is getting to the point that the server times out and only part of them get sent. This is no good. I figure, php just isn't designed to do such a task of sending so many emails.

So what I was thinking was maybe I could compile a list of addresses, subjects, and emails and put them into a file or many files or whatever and just run a quick system() call that runs a python or something script that will do the busy work without taking up too many resources and whatnot. Anybody think this is an ok idea? If so, would python be the best idea for doing this? Are there maybe any pre-made scripts that would do this for me so I don't have to rely on php to do the hard work?

Re: Mailing Script

Posted: Sun Aug 17, 2008 11:19 am
by ghurtado
It's only a good idea if you are positive that PHP is the problem, and not the mail server you are using. Are you running your own mail server?

Re: Mailing Script

Posted: Mon Aug 18, 2008 2:20 am
by shiznatix
The mail server is on the same dedicated machine. It does seam like the emails are sent, we get bounced emails and whatnot all day after sending a newsletter, but the script itself times out. I don't know how to see how many emails were sent from this script because the script times out, maybe because it is waiting for the mail server to finish but the mail server might be sending them out slowly as to not be too spammy. Since I can't see how many were sent I am just guessing that PHP is timing out and everything is sucking thus I wanted to use something that has no timeout time, like python or something.

Maybe there is something else wrong?

Re: Mailing Script

Posted: Mon Aug 18, 2008 3:06 am
by onion2k
You can't see what was sent? Doesn't the mail server have a log?

Re: Mailing Script

Posted: Mon Aug 18, 2008 3:39 am
by shiznatix
onion2k wrote:You can't see what was sent? Doesn't the mail server have a log?
this is very noobish of me but I don't know what to even be looking for in the server logs let alone be able to count mails sent out. logs don't tell me anything. Either way though, i don't want the PHP script to time out like that, i want to know after I hit "submit" how many emails will be sent out, which is doable through the python way i was thinking. Is anyone else using the swift batchSend() to send out around 10,000 emails at a time?

Re: Mailing Script

Posted: Mon Aug 18, 2008 9:03 am
by EverLearning
Better idea would be to send emails in batches of about 100, 200 emails at a time, per script execution, until all are sent. That way your script won't timeout and your mail server won't be overloaded. If you're using cron you can set your script to run every minute and to see if there are mails left to send.

Re: Mailing Script

Posted: Mon Aug 18, 2008 9:26 am
by ghurtado
Also, if you need to ensure that all emails are sent, I would recommend logging from your script either to a text file or (preferably) a database, so that even if the script is interrupted (python scripts can finish prematurely too... :) ) you will know exactly the work that was done before it aborted.

Re: Mailing Script

Posted: Mon Aug 18, 2008 10:31 am
by EverLearning
Here's how I typically handle sending large mail batches in my applications(using Swift Mailer)
  • 1) When admin clicks on "send newsletter" in admin section, script selects all mails that need to be sent and inserts them into a table "newsletter_recipients", which has fields: email and retries. I choose approach with inserting, because I collect emails from several tables in the DB, and didn't want to litter tables with additional fields whose only purpose is to reflect the sent email status.
  • 2)Cron script, which runs every minute, checks if there is anything to send. At the beginning of it, there is a check to see if the script is already running, so we don't have multiple copies of the script sending mails at once, overloading the mail server. If there are mails in the table, first 100 mails which have retries field with value less than 3(for example, you can use whatever number suits your purposes)are selected.
  • 3)Mails are then sent one by one, using $swift->send() instead of $swift->batchSend(), for greater control. If the sending is successful, email is deleted from the table, if not, retries value is incremented by 1.
  • 4)When all the mails are sent, left in the table are mails whose sending had been retried a couple of times and failed, so you can add that to some kind of report. This will just have mails that your mail server didn't accept. If you want to know what mails weren't delivered successfully you'll have to set up a return-path mail address, and then read the reports for failed deliveries from there.

Re: Mailing Script

Posted: Mon Aug 18, 2008 10:21 pm
by alex.barylski
If PHP is timing out can't you just up the execution time?

What MTA are you using? Are you sending by connecting via SMTP?

If your emails are bouncing that isn't the fault of PHP.

Re: Mailing Script

Posted: Tue Aug 19, 2008 2:18 am
by shiznatix
Hockey wrote:If PHP is timing out can't you just up the execution time?

What MTA are you using? Are you sending by connecting via SMTP?

If your emails are bouncing that isn't the fault of PHP.
Upping the execution time to like 30 minutes is a bit too much for me. I am sending via SMTP. I know the bounced emails are not PHPs fault, I am just wondering if all emails are being attempted or not.

EverLearning: sounds like a good idea. I will look into that and probably do it that way. Just seamed fancy-funtimes(R) to use a python script to do it.