Page 1 of 1

Swift mailer to send large number of emails--duplicate seen

Posted: Sat May 12, 2007 11:46 am
by minxigao
Hi,

I'm asked to develop an application that allows user to send out newsletter. Typical user will come in and send around 3000-6000 newsletter emails each time.

I have been using Swiftmailer to implement this, each time by calling the following:

$num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mynewsletter.com", "MyNewsletter"));

Where the number of email addresses in $recipients is 3000-6000.

I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the above line is called.

This has been working out fine, except lately when I start testing with real large number of emails. Last night I sent out a batch of about 3400 and called my hosting Godaddy, they told me more than 9000 are received by the server. I put some test display in the same file and I see the message is printed out 3 times. It almost looks like the php script file containing the above line is processed 3 times.

This doesn't happen all the time. Here are some scenarios:
1. Have not experience this when the number of emails is smaller, such as 200-300
2. Does not always repeat 3 times, I have seen 5,6,7 times
3. If I comment out the above $swift->batchSend line, then do the same experiment for 3400 emails, I do not see the message printed out multiple times. Only once. I repeated the experiment 4 times.

I know this sounds very wierdo. I haven't been able to google anything like this. Any suggestions? I'd really appreciate it.

Jenny

Posted: Sat May 12, 2007 1:35 pm
by Chris Corbyn
Can you post the code please? RecipientList filters out duplicates. There was a bug that did affect 3.2.0 only if you used the PHP4 version with a PHP5 installation. If you're using 3.2.0 for PHP4, on a PHP5 server, upgrade to 3.2.1. The bug caused an error handler to think there was a problem because PHP was throwing warnings about the "var" keywords and other things at it.

If you're not using 3.2.0-php4 on a PHP5 setup I'll need to see some code since this just sounds too weird :)

Posted: Sat May 12, 2007 1:52 pm
by Chris Corbyn
Actually sorry, I'm wrong there. 3.2.0 doesn't have this bug. I fixed it before I ever made an official release.

Posted: Sun May 13, 2007 3:30 am
by minxigao
In my server,the php version is php5,and i used swift 3.0.6.

Code: Select all

fwrite($handle_log,'===============beginning of log===================='.chr(10));
set_time_limit (0);//
$sendmail =& new Swift_Connection_Sendmail();
$sendmail->setTimeout(100); //100 seconds
$swift =& new Swift($sendmail);
$swift->log->enable();//to catch failed mails;
$message =& new Swift_Message($subject);
$plain_text="Hello! Please click on your attachment to view."; 
$message->attach(new Swift_Message_Part($plain_text));
$message->attach(new Swift_Message_Part($mail_contents, "text/html"));
$recipients =& new Swift_RecipientList();

$recipients->addTo($row['Email']);//got from our database,thousands of email addresses

$num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@my.com", "aaa"));
fwrite($handle_log,'=================end of log========================='.chr(10).chr(10));
In one experiment ,my log file may print log 3 or 5 or 6 times,though I access the php script file once in my browser.

Posted: Sun May 13, 2007 5:26 am
by Chris Corbyn
Is this code from a loop somewhere? If it is, can I see the loop too please?

PS: Upgrade your library. 3.0 has been done with for quite a while now ;) You can safely replace versions of Swift with new versions without worrying about having problems.

EDIT | If it's writing those log lines 5 or 6 times there, then this must be looping and in which case you're running your batch multiple times.

Posted: Wed May 16, 2007 4:17 am
by minxigao
hello,d11wtq. Thanks for all your answers before.I think more detailed problem should be listed here.
PS: Upgrade your library. 3.0 has been done with for quite a while now You can safely replace versions of Swift with new versions without worrying about having problems.
when i tried to update mail lib in my godaddy server, i met the same problem as gniq(his/her topic:problem when use latest version 3.2.1 php5).So i remained the current version 3.0 .
Here ,i post most of my code,and give my experiments.
I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the folling lines are called.

Code: Select all

fwrite($handle_log,'===============beginning of log===================='.chr(10));
$log_time = date("F j, Y, g:i a");
fwrite($handle_log,'transaction time: '.$log_time.chr(10));

require_once "../../mail_lib/Swift.php";
require_once "../../mail_lib/Swift/Connection/Sendmail.php";

fwrite($handle_log,'begin to start mail_lib ...'.chr(10));
$sendmail =new Swift_Connection_Sendmail();
$sendmail->setTimeout(100); //100 seconds
$swift = new Swift($sendmail);
//$batchMailer=new Swift_BatchMailer($swift);
$swift->log->enable();//to catch failed mails;
fwrite($handle_log,'mail_lib has beed started...'.chr(10));
		
//Create the message
$message = new Swift_Message($subject);
$plain_text="Hello! ";//change the text 
$message->attach(new Swift_Message_Part($plain_text));
$message->attach(new Swift_Message_Part($mail_contents, "text/html"));

$recipients = new Swift_RecipientList();

$batch_unit_num=300;
$batch_amount=ceil($total_mails_to_send/$batch_unit_num);
fwrite($handle_log,'total '.$total_mails_to_send.' emails to send,divided into '.$batch_amount.' batches'.chr(10));
$i=0;
$batch_index=0;
$total_mails_sent=0;
foreach ($email_list as $value)//$email_list is array and its data gets from database
{
	$recipients->addTo($value);
	$i++;
	if($i==$batch_unit_num||$batch_index>=$batch_amount)
	{
		[b]$curr_num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mysite.com", "seenname"));[/b]
		$recipients->flush(); 
		fwrite($handle_log,'  [batch '.($batch_index+1).':'.$batch_unit_num.']....succeed:'.$curr_num_sent.chr(10));//for logging
		$total_mails_sent+=$curr_num_sent;
		$i=0;
		$batch_index++;
	}
}
fwrite($handle_log,'total num sent='.$total_mails_sent.chr(10));
fwrite($handle_log,'=================end of log========================='.chr(10).chr(10));

and my log file prints the following contents:
===============beginning of log====================
transaction time: May 15, 2007, 11:58 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
===============beginning of log====================
transaction time: May 15, 2007, 11:59 pm
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================

[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================

===============beginning of log====================
transaction time: May 16, 2007, 12:00 am
begin to start mail_lib ...
mail_lib has beed started...
total 3388 emails to send,diviede into 12 batches
[batch 1:300]....succeed:300
[batch 2:300]....succeed:300
[batch 3:300]....succeed:300
[batch 4:300]....succeed:300
[batch 5:300]....succeed:300
[batch 6:300]....succeed:300
[batch 7:300]....succeed:300
[batch 8:300]....succeed:300
[batch 9:300]....succeed:299
[batch 10:300]....succeed:300
[batch 11:300]....succeed:300
total num sent=3299
=================end of log=========================
To my surprise,it prints three times ,and the two log intersect!!
But when i comment out this line:

Code: Select all

$curr_num_sent=$swift->batchSend($message, $recipients, new Swift_Address("support@mysite.com", "seenname"));
log file prints only once!
I can not explain that.I'm so confused.
[/quote]

Posted: Wed May 16, 2007 4:21 am
by minxigao
BTW,the above codes are not in a loop.

Posted: Wed May 16, 2007 9:29 am
by Chris Corbyn
Swift can't go anywhere near your calls to fwrite() so this must be something in your own logic. Is this file included into another by calling the "include" or "require" keywords? Is it possible you're including the file in multiple places?

Re: Swift mailer to send large number of emails--duplicate s

Posted: Tue Jun 26, 2007 2:53 am
by ipearx
minxigao wrote: I use PayPal Instant Payment Notification (IPN) to process the payment and when user has made the payment and PayPal verifies, the above line is called.

This has been working out fine, except lately when I start testing with real large number of emails. Last night I sent out a batch of about 3400 and called my hosting Godaddy, they told me more than 9000 are received by the server. I put some test display in the same file and I see the message is printed out 3 times. It almost looks like the php script file containing the above line is processed 3 times.
This is most likely your paypal IPN page taking too long to respond to the paypal server, so the paypal server tries again a few times until it works. You shouldn't do anything too time consuming on your IPN page. Perhaps instead of sending the emails directly from the IPN page you should flag the messages to send later, then run a PHP script via cron to process the emails every 5 minutes or so as described elsewhere on this forum.

Regards,
Tim