Swift mailer to send large number of emails--duplicate seen
Moderators: Chris Corbyn, General Moderators
Swift mailer to send large number of emails--duplicate seen
Hi,
I'm asked to develop an application that allows user to send out newsletter. Typical user will come in and send around 3000-6000 newsletter emails each time.
I have been using Swiftmailer to implement this, each time by calling the following:
$num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mynewsletter.com", "MyNewsletter"));
Where the number of email addresses in $recipients is 3000-6000.
I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the above line is called.
This has been working out fine, except lately when I start testing with real large number of emails. Last night I sent out a batch of about 3400 and called my hosting Godaddy, they told me more than 9000 are received by the server. I put some test display in the same file and I see the message is printed out 3 times. It almost looks like the php script file containing the above line is processed 3 times.
This doesn't happen all the time. Here are some scenarios:
1. Have not experience this when the number of emails is smaller, such as 200-300
2. Does not always repeat 3 times, I have seen 5,6,7 times
3. If I comment out the above $swift->batchSend line, then do the same experiment for 3400 emails, I do not see the message printed out multiple times. Only once. I repeated the experiment 4 times.
I know this sounds very wierdo. I haven't been able to google anything like this. Any suggestions? I'd really appreciate it.
Jenny
I'm asked to develop an application that allows user to send out newsletter. Typical user will come in and send around 3000-6000 newsletter emails each time.
I have been using Swiftmailer to implement this, each time by calling the following:
$num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mynewsletter.com", "MyNewsletter"));
Where the number of email addresses in $recipients is 3000-6000.
I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the above line is called.
This has been working out fine, except lately when I start testing with real large number of emails. Last night I sent out a batch of about 3400 and called my hosting Godaddy, they told me more than 9000 are received by the server. I put some test display in the same file and I see the message is printed out 3 times. It almost looks like the php script file containing the above line is processed 3 times.
This doesn't happen all the time. Here are some scenarios:
1. Have not experience this when the number of emails is smaller, such as 200-300
2. Does not always repeat 3 times, I have seen 5,6,7 times
3. If I comment out the above $swift->batchSend line, then do the same experiment for 3400 emails, I do not see the message printed out multiple times. Only once. I repeated the experiment 4 times.
I know this sounds very wierdo. I haven't been able to google anything like this. Any suggestions? I'd really appreciate it.
Jenny
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Can you post the code please? RecipientList filters out duplicates. There was a bug that did affect 3.2.0 only if you used the PHP4 version with a PHP5 installation. If you're using 3.2.0 for PHP4, on a PHP5 server, upgrade to 3.2.1. The bug caused an error handler to think there was a problem because PHP was throwing warnings about the "var" keywords and other things at it.
If you're not using 3.2.0-php4 on a PHP5 setup I'll need to see some code since this just sounds too weird
If you're not using 3.2.0-php4 on a PHP5 setup I'll need to see some code since this just sounds too weird
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
In my server,the php version is php5,and i used swift 3.0.6.
In one experiment ,my log file may print log 3 or 5 or 6 times,though I access the php script file once in my browser.
Code: Select all
fwrite($handle_log,'===============beginning of log===================='.chr(10));
set_time_limit (0);//
$sendmail =& new Swift_Connection_Sendmail();
$sendmail->setTimeout(100); //100 seconds
$swift =& new Swift($sendmail);
$swift->log->enable();//to catch failed mails;
$message =& new Swift_Message($subject);
$plain_text="Hello! Please click on your attachment to view.";
$message->attach(new Swift_Message_Part($plain_text));
$message->attach(new Swift_Message_Part($mail_contents, "text/html"));
$recipients =& new Swift_RecipientList();
$recipients->addTo($row['Email']);//got from our database,thousands of email addresses
$num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@my.com", "aaa"));
fwrite($handle_log,'=================end of log========================='.chr(10).chr(10));- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Is this code from a loop somewhere? If it is, can I see the loop too please?
PS: Upgrade your library. 3.0 has been done with for quite a while now
You can safely replace versions of Swift with new versions without worrying about having problems.
EDIT | If it's writing those log lines 5 or 6 times there, then this must be looping and in which case you're running your batch multiple times.
PS: Upgrade your library. 3.0 has been done with for quite a while now
EDIT | If it's writing those log lines 5 or 6 times there, then this must be looping and in which case you're running your batch multiple times.
hello,d11wtq. Thanks for all your answers before.I think more detailed problem should be listed here.
Here ,i post most of my code,and give my experiments.
I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the folling lines are called.
and my log file prints the following contents:
===============beginning of log====================
transaction time: May 15, 2007, 11:58 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
===============beginning of log====================
transaction time: May 15, 2007, 11:59 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
===============beginning of log====================
transaction time: May 16, 2007, 12:00 am
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
To my surprise,it prints three times ,and the two log intersect!!
But when i comment out this line:
log file prints only once!
I can not explain that.I'm so confused.
[/quote]
when i tried to update mail lib in my godaddy server, i met the same problem as gniq(his/her topic:problem when use latest version 3.2.1 php5).So i remained the current version 3.0 .PS: Upgrade your library. 3.0 has been done with for quite a while now You can safely replace versions of Swift with new versions without worrying about having problems.
Here ,i post most of my code,and give my experiments.
I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the folling lines are called.
Code: Select all
fwrite($handle_log,'===============beginning of log===================='.chr(10));
$log_time = date("F j, Y, g:i a");
fwrite($handle_log,'transaction time: '.$log_time.chr(10));
require_once "../../mail_lib/Swift.php";
require_once "../../mail_lib/Swift/Connection/Sendmail.php";
fwrite($handle_log,'begin to start mail_lib ...'.chr(10));
$sendmail =new Swift_Connection_Sendmail();
$sendmail->setTimeout(100); //100 seconds
$swift = new Swift($sendmail);
//$batchMailer=new Swift_BatchMailer($swift);
$swift->log->enable();//to catch failed mails;
fwrite($handle_log,'mail_lib has beed started...'.chr(10));
//Create the message
$message = new Swift_Message($subject);
$plain_text="Hello! ";//change the text
$message->attach(new Swift_Message_Part($plain_text));
$message->attach(new Swift_Message_Part($mail_contents, "text/html"));
$recipients = new Swift_RecipientList();
$batch_unit_num=300;
$batch_amount=ceil($total_mails_to_send/$batch_unit_num);
fwrite($handle_log,'total '.$total_mails_to_send.' emails to send,divided into '.$batch_amount.' batches'.chr(10));
$i=0;
$batch_index=0;
$total_mails_sent=0;
foreach ($email_list as $value)//$email_list is array and its data gets from database
{
$recipients->addTo($value);
$i++;
if($i==$batch_unit_num||$batch_index>=$batch_amount)
{
[b]$curr_num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mysite.com", "seenname"));[/b]
$recipients->flush();
fwrite($handle_log,' [batch '.($batch_index+1).':'.$batch_unit_num.']....succeed:'.$curr_num_sent.chr(10));//for logging
$total_mails_sent+=$curr_num_sent;
$i=0;
$batch_index++;
}
}
fwrite($handle_log,'total num sent='.$total_mails_sent.chr(10));
fwrite($handle_log,'=================end of log========================='.chr(10).chr(10));and my log file prints the following contents:
===============beginning of log====================
transaction time: May 15, 2007, 11:58 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
===============beginning of log====================
transaction time: May 15, 2007, 11:59 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
===============beginning of log====================
transaction time: May 16, 2007, 12:00 am
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
To my surprise,it prints three times ,and the two log intersect!!
But when i comment out this line:
Code: Select all
$curr_num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mysite.com", "seenname"));I can not explain that.I'm so confused.
[/quote]
- Chris Corbyn
- Breakbeat Nuttzer
- Posts: 13098
- Joined: Wed Mar 24, 2004 7:57 am
- Location: Melbourne, Australia
Re: Swift mailer to send large number of emails--duplicate s
This is most likely your paypal IPN page taking too long to respond to the paypal server, so the paypal server tries again a few times until it works. You shouldn't do anything too time consuming on your IPN page. Perhaps instead of sending the emails directly from the IPN page you should flag the messages to send later, then run a PHP script via cron to process the emails every 5 minutes or so as described elsewhere on this forum.minxigao wrote: I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the above line is called.
This has been working out fine, except lately when I start testing with real large number of emails. Last night I sent out a batch of about 3400 and called my hosting Godaddy, they told me more than 9000 are received by the server. I put some test display in the same file and I see the message is printed out 3 times. It almost looks like the php script file containing the above line is processed 3 times.
Regards,
Tim