Page 1 of 1

Sending 130.000 emails daily

Posted: Mon May 07, 2007 12:40 am
by andrys
Hi!

First of all I'd like to thank you for developing this great tool.

I am in charge of sending the daily newsletters for about 10 websites. Until now the sending was done with a loop on the classic mail() function. A complete loop took about 1 hour for a 110.000 emails list to be transfered to the server for sending. From this point the mail server takes care of sending and retrying the deferred emails.

After rebuilding the newsleter script using SwiftMailer, the whole process took about 3 hours instead of 1, with SMTP connection instead of mail(). Why is that? Shouldn't it work faster by SMTP? It is important for us that the emails get transfered to the server as fast as possible.

How can i optimize the sending process further?

Thanks,
Andrew.

Re: Sending 130.000 emails daily

Posted: Mon May 07, 2007 2:02 pm
by Chris Corbyn
andrys wrote:Hi!

First of all I'd like to thank you for developing this great tool.

I am in charge of sending the daily newsletters for about 10 websites. Until now the sending was done with a loop on the classic mail() function. A complete loop took about 1 hour for a 110.000 emails list to be transfered to the server for sending. From this point the mail server takes care of sending and retrying the deferred emails.

After rebuilding the newsleter script using SwiftMailer, the whole process took about 3 hours instead of 1, with SMTP connection instead of mail(). Why is that? Shouldn't it work faster by SMTP? It is important for us that the emails get transfered to the server as fast as possible.

How can i optimize the sending process further?

Thanks,
Andrew.
SMTP is slower. mail() just throws messages onto the spool and the messages just sit on the spool until the server can get rid of them.

EDIT | I'd love to see what sort of load average you were getting doing that 8O I'd hazard a guess that a good few emails never actually even made it to the spool neither.

Posted: Tue May 08, 2007 2:16 am
by andrys
The server is a Quad-Core and the system load peak is at ~43%.

So basically when sending directly through a SMTP server, the script waits for a response for each email?

Yesterday I tried sending a newsletter with batchSend and I got this:
PHP Fatal error: Uncaught exception 'Swift_BadResponseException' with message 'Expected response code(s) [250] but got response [421 4.4.2 http://www.domain.com Error: timeout exceeded]' in /etc/newsletter/swift/Swift.php:253
What's the correct approach? Catching the exception and disconnecting and reconnecting to the SMTP server?

Posted: Tue May 08, 2007 3:08 am
by Chris Corbyn
andrys wrote:The server is a Quad-Core and the system load peak is at ~43%.

So basically when sending directly through a SMTP server, the script waits for a response for each email?
Yes. SMTP has the benefit of receiving feedback from the server, but at a cost of time to have a conversation + of course, it has to send everything over TCP. TCP is where it slows down more than anything. If you think your emails are being sent faster with mail(), they're not. They're just being written to the disk on the server quickly. Swift does come with various connectivity options (including mail()).
Yesterday I tried sending a newsletter with batchSend and I got this:
PHP Fatal error: Uncaught exception 'Swift_BadResponseException' with message 'Expected response code(s) [250] but got response [421 4.4.2 http://www.domain.com Error: timeout exceeded]' in /etc/newsletter/swift/Swift.php:253
What's the correct approach? Catching the exception and disconnecting and reconnecting to the SMTP server?
Upgrade to 3.2.0. It won't "stop" such errors, but batchSend() recovers from them (or tries to).

That's a network issue from looking at the error message.