Time out and memory limits (ver4b3)

Swift Mailer is a fantastic library for sending email with php. Discuss this library or ask any questions about it here.

Moderators: Chris Corbyn, General Moderators

Post Reply
IshaDakota
Forum Newbie
Posts: 8
Joined: Sun Jan 18, 2009 8:01 am

Time out and memory limits (ver4b3)

Post by IshaDakota »

I am using the following code send fairly lengthy emails to up to about 1,000 recipients at a time. The process works for one or two recipients, but fails (fatal error) when it gets above about 100. At first the errors were time out variety, so I increased the max_time in php.ini. Then they were memory_limit, which I have set at 32M, and which according to what I have read, should be plenty. I must be doing something wrong.

In short, this application takes inputed html carried as a session and puts a header and footer and sends it to a list from a database (I have replaced personal information with sample text).

I am using the 4.0 beta3

Code: Select all

<?php
session_name ('sendemail');
session_start();
include ('./includes/header.html');
 
require ('./includes/mysql_connect.php'); //connects to the database
 
$additions = $_POST['additions'];
$subject = $_SESSION['subject'];
$count = 0;
 
//Include this needed file
require_once '/path/to/swift/swift_required.php';
 
// commented out because I couldn't get it to work--> require_once "/path/to/swift/classes/Swift/Plugins/AntiFloodPlugin.php";
 
//Start the mailer
$mailer = new Swift_Mailer(Swift_SendmailTransport::newInstance('/usr/sbin/sendmail -t -i'));
 
// commented out because I couldn't get it to work--> $mailer->attachPlugin(new Swift_Plugin_AntiFlood(200, 10), "anti-flood");
 
$message = Swift_Message::newInstance();
 
foreach ($additions as $value) {
$query = "SELECT email, first_name, last_name FROM fulldata WHERE briusaid='$value'";
$result = mysql_query($query) or die (mysql_error());
$email = mysql_result($result,0,"email");
$fname = mysql_result($result,0,"first_name");
$lname = mysql_result($result,0,"last_name");
 
 
$html='<html>
<table width="600" border="0">
<tr><td>
<table width="100%" border="0">
<tr>
<td>
<font face="Verdana, Arial, Helvetica, sans-serif" color="#666666" size="2"><img src="http://www.example.com/image.gif" width="150" height="47" alt="" border="0" align=""></font>
</td>
<td>&nbsp;</td>
</tr>
</table>
<p><font color="#666666" face="Verdana, Arial, Helvetica, sans-serif" size="2">Dear' .$fname. ',</p>';
 
$html.=$_SESSION['html'];
 
 
$html.='
<p>Sample closing text...</p>
  </td></tr></table>
</body></html>';
 
$no_html = strip_tags($html);
 
 
//Create a message
$message->setSubject($subject)
    ->addPart($no_html, 'text/plain')
    ->addPart($html, 'text/html')
    ->setFrom(array('me@example.com => 'My Name'))
    ->setTo(array($email => $fname. ' '. $lname));
 
//Send it
$mailer->send($message);
 
$count = $count +1;
}
 
echo "$count emails sent.";
 
mysql_close();
 
$_SESSION = array(); // Destroy the variables.
    session_destroy(); // Destroy the session itself.
    
include ('./includes/footer.html');
?>
 
Last edited by IshaDakota on Sat Jan 31, 2009 10:48 am, edited 1 time in total.
xdecock
Forum Commoner
Posts: 37
Joined: Tue Mar 18, 2008 8:16 am

Re: Time out and memory limits (ver4b3)

Post by xdecock »

for the plugins, you just have to do:

Code: Select all

$mailer->registerPlugin(new Swift_Plugin_AntiFlood(200, 10));
have you tested with another transport (smtp localhost:25)? what is the size of your html / final mail?

does this problem already exists in beta2?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

Sounds like there could be a circular reference somewhere (which would be a regression) but I'm not aware of any.

I'll look into this more later. I've beta 4 is going to be released shortly too since there's another bug that needs resolving with attachments.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

@IshaDakota, sorry I was going to do this myself but I'm flat out at work. Would you mind creating a ticket in lighthouse?

http://swiftmailer.lighthouseapp.com/
IshaDakota
Forum Newbie
Posts: 8
Joined: Sun Jan 18, 2009 8:01 am

Re: Time out and memory limits (ver4b3)

Post by IshaDakota »

Thanks.

I haven't tried Smtp Transport - I thought to try, but figured it wouldn't make a difference.

Problem did exist in beta2 as well.

I will submit a ticket.
IshaDakota
Forum Newbie
Posts: 8
Joined: Sun Jan 18, 2009 8:01 am

Re: Time out and memory limits (ver4b3)

Post by IshaDakota »

tried Smtp Transport with same results.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

I'll get onto this later and hopefully have a fix in beta 4. More than likely there's a circular reference.

However, it's also possible that the circular reference is in your own code and not in Swift Mailer. I'll find out more shortly (sorry, I'm actually at work at the moment so can't work on Swift Mailer).
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

Actually, this looks like you're using Swift Mailer incorrectly.

You've basically got this flow of execution when I break it down:

Code: Select all

$message = new Swift_Message();
 
foreach ($loadsOfRecipients as $stuff) {
  $message
    ->setSubject( ... )
    ->addPart( ... )
    ->addPart( ... )
   ;
  
  $mailer->send($message);
}
The memory leak here is in your own code (though I haven't double checked in Swift Mailer yet). This should also be doing some very horrible things that you're probably not aware of.

You're creating *one* message (that's good) before the loop. Then inside the loop you're calling addPart() twice (very bad). Calling addPart() does exactly what it says on the tin... it's "adds" a new part to the message. So in your first loop the message has 2 parts (which relates to 2 objects in memory). In the second iteration of the loop the message gets 4 parts (the original 2, plus two more). In the 100th iteration, the message will have 200 alternative bodies, which yes, probably will exhaust your memory

Basically any setter that starts with "add" actually adds to whatever is already present. Any setter that starts with "set" will override whatever is present.

Correct Pseudo-code for what you're trying to do. Two possible approaches:

1) Just create new messages inside the loop considering you're more or less rewriting the entire message anyway.
2) Use attach() and detach() to remove the old parts before adding the new ones... looks a little hairy:

Code: Select all

$message = new Swift_Message();
 
$htmlPart = null;
$textPart = null;
 
foreach ($loadsOfRecipients as $stuff) {
  if ($htmlPart) {
    $message->detach($htmlPart);
  }
  if ($textPart) {
    $message->detach($textPart);
  }
  
  $textPart = Swift_MimePart::newInstance( 'text content', 'text/plain' );
  $htmlPart = Swift_MimePart::newInstance( 'HTML content', 'text/html' );
  $message
    ->setSubject( ... )
    ->attach($htmlPart)
    ->attach($textPart)
   ;
  
  $mailer->send($message);
}
This is really my fault for not completing the documentation yet to cover the logic of attach() and detach() in any detail. In your case however I'd do what you were doing the first time, except move the message creation into the loop so you get a new message object each time:

Code: Select all

 
foreach ($loadsOfRecipients as $stuff) {
  $message = new Swift_Message();
  $message
    ->setSubject( ... )
    ->addPart( ... )
    ->addPart( ... )
   ;
  
  $mailer->send($message);
}
IshaDakota
Forum Newbie
Posts: 8
Joined: Sun Jan 18, 2009 8:01 am

Re: Time out and memory limits (ver4b3)

Post by IshaDakota »

OK, I've got it. Thanks for taking the extra time to investigate my code. I don't think you can fairly say that it's your fault that the documentation isn't complete. "Beta" means "beta," right? :wink:

One followup though, as I rewrote and tested the code a few times before I posted. In my original incarnation I had something like this (i.e. I had both the mailer and message inside the foreach loop):

Code: Select all

foreach ($loadsOfRecipients as $stuff) {
 
require '/path/to/swift/swift_required.php';
 
$mailer = new Swift_Mailer(Swift_SendmailTransport::newInstance('/usr/sbin/sendmail -t -i'));
 
$message = new Swift_Message();
$message
   ->setSubject( ... )
   ->addPart( ... )
   ->addPart( ... )
   ;
  
$mailer->send($message);
}
Clearly this is wrong. It should also lead to memory and time out errors, correct?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

Yeah don't put *everything* in the loop. By placing the mailer creation in the loop you're actually opening numerous connections to the SMTP server :)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

Please also update to beta-4... there was a major bug fix that would have affected you (and almost anybody using it).

http://swiftmailer.org/betas/Swift-4.0.0-b4.tar.gz

Oh, the joys of using beta :)
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Re: Time out and memory limits (ver4b3)

Post by Chris Corbyn »

IshaDakota wrote:OK, I've got it. Thanks for taking the extra time to investigate my code. I don't think you can fairly say that it's your fault that the documentation isn't complete. "Beta" means "beta," right? :wink:

One followup though, as I rewrote and tested the code a few times before I posted. In my original incarnation I had something like this (i.e. I had both the mailer and message inside the foreach loop):

Code: Select all

foreach ($loadsOfRecipients as $stuff) {
 
require '/path/to/swift/swift_required.php';
 
$mailer = new Swift_Mailer(Swift_SendmailTransport::newInstance('/usr/sbin/sendmail -t -i'));
 
$message = new Swift_Message();
$message
   ->setSubject( ... )
   ->addPart( ... )
   ->addPart( ... )
   ;
  
$mailer->send($message);
}
Clearly this is wrong. It should also lead to memory and time out errors, correct?
Sorry for multiple replies here. Although this code is wrong, it should not give memory errors. Timeouts cannot be avoided since PHP will use a 30 second timeout by default (you need to use set_time_limit(0) to remove it. If you're getting memory errors from this code then I will have to investigate my code further. Timeouts normal though.
Post Reply