Page 1 of 2

Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 3:59 pm
by bozzaj
I'm looking into switching from PHPMailer to Swift (testing with 4.0b5) and I think I'm running into a serious speed issue. I ran some profilers on my code and I believe I've narrowed it down to QpEncoder. Basically, here are my two scripts:

PHPMailer

Code: Select all

$num = 50;
 
$PHPMAILDIR = "/home/www/msg_intranet/php/phpmail/";
require_once($PHPMAILDIR . "class.phpmailer.php");
 
$mail = new phpmailer();
$mail->PluginDir = $PHPMAILDIR;
$mail->IsSMTP();
$mail->Host = "localhost";
$mail->SMTPKeepAlive = true;
$mail->SetLanguage("en", $PHPMAILDIR);
$mail->Timeout = 60;
$mail->Encoding = "quoted-printable";
$mail->CharSet = 'us-ascii';
 
$sender = 'test@domain.com';
$mail->From = $sender;
$mail->Subject = 'Test Subject';
 
$body = file_get_contents('htmlbody.html');
 
$altbody = <<<EOT
This is the text body.
EOT;
 
$mail->IsHTML(true);
$mail->Body = $body;
$mail->AltBody = $altbody;
 
$mail->Sender = $sender;
$mail->ReturnPath = $sender;
 
$timein = time();
for ($i=0;$i<$num;$i++) {
  $to = "speedtest+$i@domain.com";
  $mail->ClearAllRecipients();
  $mail->AddAddress($to);
  $mail->Send();
}
$timeout = time();
 
print "Done - ".($timeout-$timein)." seconds\n";
 
Swift Mailer - using send()

Code: Select all

$num = 50;
 
$SWIFTMAILDIR = "/home/www/msg_intranet/php/swift/";
require_once($SWIFTMAILDIR . "swift_required.php");
 
$mailtransport = Swift_SmtpTransport::newInstance('localhost', 25);
$mailer = Swift_Mailer::newInstance($mailtransport);
 
$sender = 'test@domain.com';
$mail = new Swift_Message();
$mail->setFrom($sender);
$mail->setSubject('Test Subject');
$mail->setCharset('us-ascii');
 
$body = file_get_contents('htmlbody.html');
 
$altbody = <<<EOT
This is the text body.
EOT;
 
$mail->setBody($body, 'text/html');
$mail->addPart($altbody, 'text/plain');
 
$mail->setSender($sender);
$mail->setReturnPath($sender);
 
 
$timein = time();
for ($i=0;$i<$num;$i++) {
  $to = "speedtest+$i@domain.com";
  $mail->setTo(array($to));
  $mailer->send($mail);
}
$timeout = time();
 
print "Done - ".($timeout-$timein)." seconds\n";
 
The file htmlbody.html is 31229 bytes, fairly typical HTML, 932 lines quite a few over 76 characters.

Output from PHPMailer Version:
Done - 10 seconds

Output from Swift Version:
Done - 33 seconds

This is sending 50 emails. Increasing it up to 500 emails changes it to 98 seconds and 331 seconds, so the timing is fairly consistent at a 3.3:1 ratio. I tried tweaking the byte stream (from 4 to 8 then higher to 128+), but it would only drop Swift's speed down to 23 seconds at the minimum.

My question - Is this something that's currently being worked on? For single emails and such, this isn't too much of a big deal, but for newsletters and other larger mailings, this becomes a breaking point. For instance, 10000 messages (at these rates) would be 33 minutes or so with PHPMailer and almost 2 hours with Swift.

I'm going to look further into QPEncoder to see if I can do some other tweaks, but I wanted to first see if anyone else has any ideas first.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 4:06 pm
by John Cartwright
It would be interesting to see the bencharks for v4. Although I never benchmarked our programs using v3, I have noticed a significant drop in performance.

We send 400 emails every few hours (all exact emails, i.e. no customization) and it takes several minutes.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 4:16 pm
by xdecock
I've noted a notable performance drop also, that's why i'm on the project, try a beta2 release, and you'll probably note that there are major improvements.

performance tweaking is quite huge works in fact I'm currently using the test-suite to run some profiling, if you have some performance test-suite to propose, i'll be glad to take thoses as basis for optimizations works.

as i always say, optimization are not needed everywhere, so let me know where you needs it

(PS: i'll see if i have more pending optimizations to commit)

PS: with the V3 i'm able to sustain 240K Mails/hour on a single system (won't say how, i have to switch to V4 for some reasons, and can't justify any performances regressions to my clients, so performances is a key point, and i'm working on it)

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 4:24 pm
by bozzaj
Jcart wrote:It would be interesting to see the bencharks for v4. Although I never benchmarked our programs using v3, I have noticed a significant drop in performance.

We send 400 emails every few hours (all exact emails, i.e. no customization) and it takes several minutes.
I started researching the issue when I found that 7200 or so emails that would normally take around 14 minutes ended up taking around 57 minutes using Swift v4. Even using batchsend() didn't help. Right now I'm going to stick with PHPMailer until I have time to mess with the QPEncoder code to see if I can increase the speed of things.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 4:36 pm
by bozzaj
xdecock wrote:I've noted a notable performance drop also, that's why i'm on the project, try a beta2 release, and you'll probably note that there are major improvements.

performance tweaking is quite huge works in fact I'm currently using the test-suite to run some profiling, if you have some performance test-suite to propose, i'll be glad to take thoses as basis for optimizations works.

as i always say, optimization are not needed everywhere, so let me know where you needs it

(PS: i'll see if i have more pending optimizations to commit)

PS: with the V3 i'm able to sustain 240K Mails/hour on a single system (won't say how, i have to switch to V4 for some reasons, and can't justify any performances regressions to my clients, so performances is a key point, and i'm working on it)
I'm confused - You say to try a beta2 release. I'm using the latest beta of 4.0.0 (4.0.0 Beta 5 posted today). Are you saying to move back to beta 2?

The boxes I'm using to send email aren't powerful by any means, so I'm sure I could build a new server and/or move to a threading model, but I was hoping for just a drop-in replacement.

I changed the htmlbody to be nothing more than <b>HTML</b>. At 500 emails, PHPMailer was 52 seconds, Swift v4 was 55 seconds. So, basically, eliminating the larger HTML to encode allows the two to run close to the same speed.

I think I'll try the latest v3 to see what the difference is.

I could probably whip up a quick test - It would be more the size of the HTML to encode over anything else. The larger the body is, the longer the encoding takes.

Thanks for your help.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 4:42 pm
by xdecock
No, i said that since beta2 we've got huge speed improvements.

I'm currently working on a new way to apply lightspeed qpencoding, don't expect till 2-3 AM tought :p (more realistic delays 'll be something like 2 days)

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 5:43 pm
by Chris Corbyn
This is a concern. I believe the problem can probably be narrowed down to a lower level than QPEncoder and more to the CharacterReaders/Streams. I was aware that there would be a drop in performance due to the i18n compatibility and full RFC 2045 compliance but it need not be this slow.

We'll invest some time in this. I should mention that PHPMailer's QP encoding algorithm is broken and will lead to corruption in some cases when used with UTF-8. It's also not RFC 2045 compliant so may lead to spam detection at times. It is faster though since they just very roughly encode the data using a Regex, the way Swift Mailer v3 used to do it. But this causes issues, hence the complete i18n support.

We hear ya, and we'll improve it. Right now it does not make use of the mb_* functions. I've created a ticket to investigate the use of these to speed things up and I believe that will add marked performance improvements.

Please do remember this is still beta.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 6:00 pm
by bozzaj
xdecock wrote:No, i said that since beta2 we've got huge speed improvements.

I'm currently working on a new way to apply lightspeed qpencoding, don't expect till 2-3 AM tought :p (more realistic delays 'll be something like 2 days)
Ok, now I understand. :) Glad to see activity.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 6:45 pm
by bozzaj
Chris Corbyn wrote:This is a concern. I believe the problem can probably be narrowed down to a lower level than QPEncoder and more to the CharacterReaders/Streams. I was aware that there would be a drop in performance due to the i18n compatibility and full RFC 2045 compliance but it need not be this slow.
That would explain the change when I adjusted the size send to readBytes(). It didn't really change too much though, even when I greatly increased the size. Wouldn't that eliminate those issues? Granted it would break the maxlinelength, but if I read the entire body in as one sequence, it was still slow.
Chris Corbyn wrote:We'll invest some time in this. I should mention that PHPMailer's QP encoding algorithm is broken and will lead to corruption in some cases when used with UTF-8. It's also not RFC 2045 compliant so may lead to spam detection at times. It is faster though since they just very roughly encode the data using a Regex, the way Swift Mailer v3 used to do it. But this causes issues, hence the complete i18n support.
This was one reason I wanted to switch - Swift seems to be much more concerned with compliancy. But, actually, the Regex QP Encoding from PHPMailer was replaced over a year ago. That one was probably about 20-30% faster than the current, but definitely broken. Looking at the current EncodeQP() function, it seems to be quite similar in style, checking each character individually, but it does some things differently. I couldn't tell you if it would be RFC 2045 compliant, and it make some assumptions, but overall the ideas between the two are quite similar.
Chris Corbyn wrote:We hear ya, and we'll improve it. Right now it does not make use of the mb_* functions. I've created a ticket to investigate the use of these to speed things up and I believe that will add marked performance improvements.

Please do remember this is still beta.
Don't worry, I understand it's still beta. I just wanted to note my experiences and see if the issue was known.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 9:43 pm
by Chris Corbyn
I'll take a look at PHPMailer's algorithm now and see what's being done. Pretty much my way of doing things was to translate the RFC into a set of unit tests and code around those.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Thu Feb 05, 2009 9:45 pm
by Chris Corbyn
Actually... before we put this down entirely to QPEncoder, could you please try something for me and let me know if there are differences in speed? :)

Put this right after your include for swift_required.php:

Code: Select all

Swift_Preferences::getInstance()->setCacheType('array');
I'm wondering if disk I/O could be slowing things down (QPEncoder writes to disk if disk caching is enabled).

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Fri Feb 06, 2009 4:56 am
by xdecock
Following my profile,

The performances bottlenecks are (on Swift_Mime_ContentEncoder_QpContentEncoderAcceptanceTest)

In red every function over .3s needing some tuning.

2.87 => ArrayCharacterStream
* 2.7 => importByteStream
* 0.16 => readByte
* 0.12 => flushContent
0.55 => QPContentEncoder
* 0.55 => encodeByteStream
0.53 => ArrayByteStream
* 0.45 => read
* 0.08 => write
0.29 => php:internal
* 0.09 call_user_func_array
* 0.04 str_split
* 0.02 array_unshift
* 0.02 file_get_contents
0.26 => QPEncoder
* 0.10 _nextSequence
* 0.76 _encodeByteSequence
* 0.75 _standardize
0.17 => ByteArrayReplacementsFilters
* 0.12 filter
* 0.05 shouldBuffer

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Fri Feb 06, 2009 8:06 am
by bozzaj
Chris Corbyn wrote:

Code: Select all

Swift_Preferences::getInstance()->setCacheType('array');
I'm wondering if disk I/O could be slowing things down (QPEncoder writes to disk if disk caching is enabled).
100 emails, with array cache: 65 seconds
100 emails, without array cache: 66 seconds

Ran it a few times and came up with the same numbers.

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Fri Feb 06, 2009 8:49 am
by xdecock
Ok, i'm on the point to commit some major changes in CharacterStream

New values:

0.75 => ArrayCharacterStream
* 0.58 => importByteStream

* 0.15 => readByte
* 0.11 => flushContent
0.372 => QPContentEncoder
* 0.37 => encodeByteStream

0.312 => ArrayByteStream
* 0.24 => read
* 0.08 => write

0.25 => QPEncoder
* 0.10 _nextSequence
* 0.78 _encodeByteSequence
* 0.72 _standardize
0.17 => ByteArrayReplacementsFilters
* 0.12 filter
* 0.05 shouldBuffer
0.17 => php:internal
* 0.09 call_user_func_array
* 0.04 str_split
* 0.02 array_unshift
* 0.02 file_get_contents

Re: Switching from PHPMailer to Swift - Speed Issues

Posted: Fri Feb 06, 2009 8:59 am
by xdecock
Commited: http://github.com/swiftmailer/swiftmail ... d58379d650

if you want to dl the patched version for testing: http://github.com/swiftmailer/swiftmail ... d58379d650

Let us know the new performance