Page 1 of 1

Swift mailer and UTF

Posted: Wed Jun 13, 2007 12:44 pm
by adepali
Weirdan | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]


I have trouble sending Greek mail using UTF-8 encoding. I tried several solutions with different levels of success, but I always had problems. The latest try was the Swift mailer, here's some relative code:

Code: Select all

function xmail2($email_address, $email_from, $subject, $msg, $attach_filepath, $email_address_cc="")
{
               $list_to = explode( ',', $email_address );
                $list_cc = explode( ',', $email_address_cc );
             $mailer =& new Swift( new Swift_Connection_NativeMail() );
                $message =& new Swift_Message( $subject, '', "text/plain", "8bit", "utf-8" );
                $recipients =& new Swift_RecipientList();
                foreach ( $list_to as $to )
                {
                        $to = trim( $to );
                        if ( $to ) $recipients->addTo( $to );
                }
                foreach ( $list_cc as $to )
                {
                        $to = trim( $to );
                        if ( $to ) $recipients->addCC( $to );
                }
              $from =& new Swift_Address( "mymail@me.gr", "My name" );
                $part = new Swift_Message_Part( $msg, "text/plain", "8bit", "utf-8" );
                $message->attach( $part );

                foreach ( $attach_filepath as $attachment )
                {
                        $message->attach( new Swift_Message_Attachment( new Swift_File( $attachment ) ) );
                }
                return $mailer->send( $message, $recipients, $from );
}
However my messages are sent with 7bit encoding in the headers, and sometimes (rarely, but still) some UTF characters are corrupted. Am I doing something wrong? Any suggestions for some mail library that seamlessly supports UTF8?


Weirdan | Please use

Code: Select all

,

Code: Select all

and [syntax="..."] tags where appropriate when posting code. Your post has been edited to reflect how we'd like it posted. Please read:  [url=http://forums.devnetwork.net/viewtopic.php?t=21171]Posting Code in the Forums[/url] to learn how to do it too.[/color]

Posted: Wed Jun 13, 2007 2:41 pm
by Chris Corbyn
Swift supports UTF-8. It's most likely that one of the intermediate servers doesn't. Swift has to use QP encoding on the UTF-8 in order to get it to transmit over SMTP (no 8-bit data allowed in SMTP by default!) but it should still arrive and be displayed as UTF-8. What mail client are you using? Hotmail and Yahoo! do not display UTF-8 correctly just in case you're viewing it in one of those.

Posted: Thu Jun 14, 2007 1:13 am
by adepali
I'm mostly using Thunderbird, but I checked with Yahoo webmail and the corrupted characters were the same. However I was a bit surprised at these headers of the received mail:

Content-Type: multipart/alternative; boundary="_=_swift-186963058046702a7ef2d0a4.02950235_=_"
Content-Transfer-Encoding: 7bit

As an additional note, sending a utf mail using a mail client from the same machine to the same email address works just fine, I only have the problem when the mail is sent through the PHP application

Maybe I should also note that I'm using the latest version of Swift (June 10)

Posted: Thu Jun 14, 2007 1:29 am
by feyd
Are the encodings set on the pages you are accessing also correct accordingly?

Posted: Tue Jun 19, 2007 9:55 am
by adepali
The encoding is right on the page, printing the body text on the browser works fine. However additional checks showed that there's a pattern to the corrupted utf string: if a sentence (\n terminated string) in UTF-8 is longer than 1988 bytes (bytes, not necessarily multibyte characters), then the character at the 1988 byte gets corrupted. Note that if there are several sentences whose size is less than 1988 there's no corruption going on, even if the entire body is of much greater length. Maybe this helps you?

Posted: Tue Jun 19, 2007 4:09 pm
by Chris Corbyn
You're actually doing this wrong in the code which may be screwing things up too.

Code: Select all

function xmail2($email_address, $email_from, $subject, $msg, $attach_filepath, $email_address_cc="") 
{ 
               $list_to = explode( ',', $email_address ); 
                $list_cc = explode( ',', $email_address_cc ); 
             $mailer =& new Swift( new Swift_Connection_NativeMail() ); 
                $message =& new Swift_Message( $subject); 
                $recipients =& new Swift_RecipientList(); 
                foreach ( $list_to as $to ) 
                { 
                        $to = trim( $to ); 
                        if ( $to ) $recipients->addTo( $to ); 
                } 
                foreach ( $list_cc as $to ) 
                { 
                        $to = trim( $to ); 
                        if ( $to ) $recipients->addCC( $to ); 
                } 
              $from =& new Swift_Address( "mymail@me.gr", "My name" ); 
                $part = new Swift_Message_Part( $msg, "text/plain", "8bit", "utf-8" ); 
                $message->attach( $part ); 

                foreach ( $attach_filepath as $attachment ) 
                { 
                        $message->attach( new Swift_Message_Attachment( new Swift_File( $attachment ) ) ); 
                } 
                return $mailer->send( $message, $recipients, $from ); 
}
Forcing 8-bit encoding will cause problems too. Swift will use QP encoding by default so hat all intermediate servers will not choke on the non-ascii data. So my final version of your function would look like this:

Code: Select all

function xmail2($email_address, $email_from, $subject, $msg, $attach_filepath, $email_address_cc="") 
{ 
               $list_to = explode( ',', $email_address ); 
                $list_cc = explode( ',', $email_address_cc ); 
             $mailer =& new Swift( new Swift_Connection_NativeMail() ); 
                $message =& new Swift_Message( $subject ); 
                $recipients =& new Swift_RecipientList(); 
                foreach ( $list_to as $to ) 
                { 
                        $to = trim( $to ); 
                        if ( $to ) $recipients->addTo( $to ); 
                } 
                foreach ( $list_cc as $to ) 
                { 
                        $to = trim( $to ); 
                        if ( $to ) $recipients->addCC( $to ); 
                } 
              $from =& new Swift_Address( "mymail@me.gr", "My name" ); 
                $part = new Swift_Message_Part( $msg, "text/plain" ); 
                $part->setCharset("utf-8");
                $message->attach( $part ); 

                foreach ( $attach_filepath as $attachment ) 
                { 
                        $message->attach( new Swift_Message_Attachment( new Swift_File( $attachment ) ) ); 
                } 
                return $mailer->send( $message, $recipients, $from ); 
}
Also, for future reference, Swift detects the UTF-8 itself if you dont specify it :)

Posted: Wed Jun 20, 2007 4:38 am
by adepali
Thanks for your reply. I tried that before switching to the code I posted here, but it only sends the first couple of lines of the message. Whenever the encoding is NOT specified in the constructor I get the same: a few lines of message body, with an extra line between them that shouldn't be there in the first place, and that's it.
Using '7bit' and '8bit' in the constructor both send the mail, but both cause the corruption I described.

If there's a way to attach files here I can attach a sample of the mail I'm trying to send...

Posted: Fri Jun 22, 2007 5:41 am
by adepali
Ok I figured what the problem is: According to RFC 2822 that sendmail apparently confronts to, line length for each mail may not exceed 998 characters. These characters, plus the trailing new line, times 2 (for multibyte utf-8) is the 1998 bytes where corruption is observed according to my previous mail.

I will change my program to add some new lines in the message body to avoid this, but I think this should be the library's work. Mail clients (such as Thunderbird, Outlook etc) automatically break long lines to avoid this issue.