Cut off Emails

Swift Mailer is a fantastic library for sending email with php. Discuss this library or ask any questions about it here.

Moderators: Chris Corbyn, General Moderators

aedmonds
Forum Newbie
Posts: 6
Joined: Thu Jul 26, 2007 3:40 pm

Cut off Emails

Post by aedmonds »

I'm running into a consistent problem with emails sent out with SwiftMailer being cut off. It's consistent because it is always cut off where a period (.) should be. I'm wondering what can be done to fix this (not saying the original developers have to do all the digging, I am Zend certified so I can contribute if I know where to look).

Anyways, the system I built was a custom clone of the popular MailMan Mailing list system. It uses PHP5, MySQL, and cron. There is a custom class the grabs the emails sent to a specific address on a POP3 server and inserts the raw data into the database. It also parses the raw data and splits out all mail parts including attachments (it saves the attachments to the server filesystem outside the web root). The message is placed in a queue that waits for moderator approval of the message. The cron job I have checks for approved messages and re-constructs the message using Swift_Message_Part, Swift_Message_EmbeddedFile, and Swift_Message_Attachment.

The system works excellent (it's been able to send crazy emails with various message parts and files attached), but every once in a while a message gets sent that is truncated always where a period should but it must also crash the script because it never gets marked as complete and those signed up to receive the email never see it.

I know there was another thread here where someone was having the same issues but I don't think it was ever resolved. It would be great if I could fix this so I won't always have to be worried about an email getting jammed in the queue.

Here is an example:

Originally Sent Message

Code: Select all

Delivered-To: announcements@example.com
Received: by 10.100.47.16 with SMTP id u16cs359500anu;        Thu, 28 Jun 2007 10:32:40 -0700 (PDT)
Received: by 10.100.140.6 with SMTP id n6mr1411312and.1183051960223;        Thu, 28 Jun 2007 10:32:40 -0700 (PDT)
Return-Path: <user@example.com>
Received: from alnrmhc15.comcast.net (alnrmhc15.comcast.net [204.127.225.95])        by mx.google.com with ESMTP id c9si11689731ana.2007.06.28.10.32.39;        Thu, 28 Jun 2007 10:32:40 -0700 (PDT)
Received-SPF: pass (google.com: best guess record for domain of user@example.com designates 204.127.225.95 as permitted sender)
Received: from lori1e24eb7a86 (c-XX-XX-XX-XX.hsd1.wa.comcast.net[XX.XX.XX.XX])          by comcast.net (alnrmhc15) with SMTP          id <20070628173238b1500cfu0ae>; Thu, 28 Jun 2007 17:32:39 +0000
Reply-To: <user@example.com>
From: "Nolan & Lori Host" <user@example.com>
To: <announcements@example.com>
Subject: Want to Buy - Homeschool Curriculum
Date: Thu, 28 Jun 2007 10:32:39 -0700
Message-ID: <000001c7b9aa$535affc0$6400a8c0@lori1e24eb7a86>
MIME-Version: 1.0
Content-Type: multipart/alternative;	boundary="----=_NextPart_000_0001_01C7B96F.A6FC27C0"
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.6822
Importance: Normal
Thread-Index: Ace5qlLiDzJwq9tHRgiSd/+n1VGwgw==
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138

This is a multi-part message in MIME format.

------=_NextPart_000_0001_01C7B96F.A6FC27C0
Content-Type: text/plain;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Dear HOFCC Families,

=20

I am having trouble finding a few expensive curriculum items used. So, I
thought I'd check with all of you.  I would like to buy the following =
items:

=20

*	Teaching Textbooks Pre-Algebra complete set
*	Apologia: Exploring Creation with Physical Science - Complete Course
on CD-ROM, Version 8.0

=20

If you have these in nice condition and wish to sell them, I'm =
interested.

=20

Thank you and God's Blessings to you,

Lori Host


------=_NextPart_000_0001_01C7B96F.A6FC27C0
Content-Type: text/html;
	charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html>

<head>
<META HTTP-EQUIV=3D"Content-Type" CONTENT=3D"text/html; =
charset=3Dus-ascii">


<meta name=3DGenerator content=3D"Microsoft Word 10 (filtered)">

<style>
<!--
 /* Font Definitions */
 @font-face
	{font-family:Wingdings;
	panose-1:5 0 0 0 0 0 0 0 0 0;}
 /* Style Definitions */
 p.MsoNormal, li.MsoNormal, div.MsoNormal
	{margin:0in;
	margin-bottom:.0001pt;
	font-size:11.0pt;
	font-family:Arial;}
a:link, span.MsoHyperlink
	{color:blue;
	text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
	{color:purple;
	text-decoration:underline;}
span.EmailStyle17
	{font-family:Arial;
	color:windowtext;}
@page Section1
	{size:8.5in 11.0in;
	margin:.5in 1.0in .55in 1.0in;}
div.Section1
	{page:Section1;}
 /* List Definitions */
 ol
	{margin-bottom:0in;}
ul
	{margin-bottom:0in;}
-->
</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>Dear
HOFCC Families,</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>I am
having trouble finding a few expensive curriculum items used. So, I =
thought I’d
check with all of you. &nbsp;I would like to buy the following =
items:</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>&nbsp;</span></font></p>

<ul type=3Ddisc>
 <li class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>Teaching
     Textbooks Pre-Algebra complete set</span></font></li>
 <li class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>Apologia:
     Exploring Creation with Physical Science – Complete Course on
     CD-ROM, Version 8.0</span></font></li>
</ul>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>If
you have these in nice condition and wish to sell them, I’m =
interested.</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>&nbsp;</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>Thank
you and God’s Blessings to you,</span></font></p>

<p class=3DMsoNormal><font size=3D2 face=3DArial><span =
style=3D'font-size:10.0pt'>Lori
Host</span></font></p>

</div>

</body>

</html>

------=_NextPart_000_0001_01C7B96F.A6FC27C0--


Received Message sent using Swift:

Code: Select all

Delivered-To: aedmonds@example.com
Received: by 10.114.209.4 with SMTP id h4cs293458wag;
        Thu, 16 Aug 2007 15:50:50 -0700 (PDT)
Received: by 10.115.89.1 with SMTP id r1mr2501710wal.1187304650227;
        Thu, 16 Aug 2007 15:50:50 -0700 (PDT)
Return-Path: <mailer@example.com>
Received: from spunkymail-a5.g.dreamhost.com (sd-green-bigip-74.dreamhost.com [208.97.132.74])
        by mx.google.com with ESMTP id l36si2006774waf.2007.08.16.15.50.49;
        Thu, 16 Aug 2007 15:50:50 -0700 (PDT)
Received-SPF: neutral (google.com: 208.97.132.74 is neither permitted nor denied by best guess record for domain of mailer@example.com) client-ip=208.97.132.74;
Authentication-Results: mx.google.com; spf=neutral (google.com: 208.97.132.74 is neither permitted nor denied by best guess record for domain of mailer@example.com) smtp.mail=mailer@example.com
Received: from SwiftUser (runts.dreamhost.com [208.113.138.13])
	by spunkymail-a5.g.dreamhost.com (Postfix) with ESMTP id 4289914D6B1
	for <aedmonds@example.com>; Thu, 16 Aug 2007 15:50:47 -0700 (PDT)
To: aedmonds@example.com
From: mailer@example.com
Reply-To: mailer@example.com
Subject: Want to Buy - Homeschool Curriculum
Date: Thu, 16 Aug 2007 15:50:48 -0700
X-Mailer: Swift 3.1.5
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="_=_swift-20906150646c4d4c948cc80.77865094_=_"
Content-Transfer-Encoding: 7bit
Sender: mailer@example.com
Errors-To: mailer@example.com
Message-ID: <2147483647.27010.1906982932.swift@runts>

This is a message in multipart MIME format.  Your mail client should not
be displaying this. Consider upgrading your mail client to view this
message correctly.
--_=_swift-20906150646c4d4c948cc80.77865094_=_
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Dear HOFCC Families,

=


=
I am having trouble finding a few expensive curriculum items used. So, I
=
thought I'd check with all of you.  I would like to buy the following ite=
ms:

=


=
*	Teaching Textbooks Pre-Algebra complete set
*	=
Apologia: Exploring Creation with Physical Science - Complete Course
on=
 CD-ROM, Version 8.0

=


=
If you have these in nice condition and wish to sell them, I'm interested=
I have probably 8 other emails that have the same problem.

Any insight would be appreciated.

-Aaron

EDIT: Oh and I'm using 3.1.5 but I've also tested this using 3.2.5 and 3.2.6.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Bailing on periods makes sense from a possible bug point of view since "." by itself (on it's own line) means "end of email", but I do have code to check for this. I'll look into it today.

Other people have had this problem where a NUL byte occurred in the text they were tryin to send (invalid ascii).

viewtopic.php?t=68249&start=15
japanitrat
Forum Newbie
Posts: 12
Joined: Sat May 12, 2007 10:34 pm

Post by japanitrat »

hi, i am encountering the same problem.

it only occurs, if I try to send a mail with UTF-8 characters from php4 (some server).
sending the same mail from php5 (from my localhost) works seamlessy.
utf8_decode before sending the mail does not help anything.

The template, that is used to send works on PHP4, if no special character like ä,ü,ö (german umlauts) are in the replaced content.
but if there is only one these characters, the whole mail cuts off with = in different places.

so i dont think, that this is a problem with especially NUL bytes. it seems a general php4 swiftmailer problem with special characters.


It would be great to get a solution to this, since I can't just put the mailer on a PHP5 server, nor I can update the php4 server to php5 :(
japanitrat
Forum Newbie
Posts: 12
Joined: Sat May 12, 2007 10:34 pm

Post by japanitrat »

update:

its not a swiftmailer-only-problem.

i just replaced the defined SMTP connection on the mentioned PHP4-server to another SMTP server where i got access to, and tadaaaaa: no email is being cut off.

it really seems to be a swiftmailer-to-SMTP-server-on-PHP4-thing, where some characters may lead to to corrupted emails on some servers.



now, get your brains on :)
chuckl
Forum Commoner
Posts: 61
Joined: Wed May 23, 2007 7:36 am

Post by chuckl »

Japanitrat,

Could you clarify something for me please?
If I understand you correctly a particular combination of Swiftmailer, php4 and a mailserver is cutting off the emails.
However, when you change the mailserver to which Swiftmailer is connecting to a different server, but leave everything else the same, the mail goes through perfectly?

If so, that would suggest to me that it is a mailserver quirk/setup problem.

Do you know what mail software the mailserver is running?
japanitrat
Forum Newbie
Posts: 12
Joined: Sat May 12, 2007 10:34 pm

Post by japanitrat »

add: i just tried the smtp-server (the one, which breaks the mail) on my localhost (PHP5), and it corrupts the mail, too, but it stays readable in my thunderbird (although the source of the mail is b0rken). so it's not PHP4 only, its PHP5, too, but less messy.


i try to sum up:

1. SMTP Server A (i found out, it's a qmail-smtpd 1.0.3 with tls+smtp_auth) sends all UTF-8 mails without cutting off in PHP4 and PHP5

2. SMTP Server B (which configs i do not know, but it's a big-company-server, and i mean really BIG) sends broken UTF-8 mails, while PHP4-generated mails are not readable at all (php5 ones stay renderable, but have broken sources, too


uhmmm, did i forgot something?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

What you're saying leads me to believe there are some corrupt byte sequences in the UTF-8 japanitrat. Could you send me a sample of something that's breaking? (Not a copy/paste though... something attached to an email which I can examine).
japanitrat
Forum Newbie
Posts: 12
Joined: Sat May 12, 2007 10:34 pm

Post by japanitrat »

i believe that, too, since i encountered a similar problem with PEAR::XML_Serializer and UTF-8 encoding.
There I am parsing form data from a UTF-8 encoded XHTML document form (also with accept-charset="utf-8" and desired PHP headers charset=utf-8) with umlauts and delivering that to another server via XML_Serializer and Unserializer.

But the Unserializer throws an error, that the XML input is broken, although it looks okay in a log-file. When I make use of the encodeFunction and decodeFunction options in (Un)Serializer, where all data is completely encoded to UTF-8 with utf8_encode() and decoded with utf8_decode(), no error occurs. this also means, that already encoded data will be double-encoded (which is okay on my current XML-application, since it will always completely decode when receiving the data).

this leads me to the conclusion that there are still characters in my message that are not proper UTF-8, that XML_Unserializer causes to throw the error. This could also be the reason why some SMTP servers do corrupt the message ... but i am not sure about that, since i could not exactly localize those charaters ....

i am going to send you emails via both servers (if i dont get banned before)-

[edit] look into your forum inbox [/edit]
Last edited by japanitrat on Fri Aug 17, 2007 11:24 am, edited 2 times in total.
japanitrat
Forum Newbie
Posts: 12
Joined: Sat May 12, 2007 10:34 pm

Post by japanitrat »

one small thing i saw in the headers:

the Content-Transfer-Encoding of the custom message parts on the SMTP-Server that cuts the mails are "quoted-printable", while the one, which sends fine emails are "8bit".

i did not declare them manually.
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Guys, could you download version 3.3.1 and let me know if this fixes the problem please? :)
CRichardson
Forum Newbie
Posts: 12
Joined: Sat Jun 02, 2007 7:22 pm

Post by CRichardson »

d11wtq wrote:Guys, could you download version 3.3.1 and let me know if this fixes the problem please? :)
Thanks, uploading this now. Will let you know within the next 24 hours :)
chuckl
Forum Commoner
Posts: 61
Joined: Wed May 23, 2007 7:36 am

Post by chuckl »

3.3.1 passes part 1 of 'class killer email I have known', which is, use sendmail, and start and end the email with 2 or 3 lines of single '.' characters and sprinkle a couple in the middle i.e.

.
.
.
part message body
.
.
.
part message body
.
.
.

That killed 3.3.0, whole message truncated, so a good start. Nice detective work Chris.

Rgds,
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

~chuckl, does your post roughly translate to "3.3.1 works where 3.3.0 didn't"? Or did you imply that "3.3.1 works better, but it's still not right"? :)

Thanks for testing by the way.
chuckl
Forum Commoner
Posts: 61
Joined: Wed May 23, 2007 7:36 am

Post by chuckl »

It translates to a very emphatic - mail sent using 3.3.0 and sendmail would arrive blank - truncated after the header.
Switching to 3.3.1 it arrives perfect in all it's pristine glory. Suprised you didn't hear the loud whoops from Canada and West London in Manchester.

So far I've tested sendmail - server default (-t -i), sendmail Swift (-bs), smtpauth and smtp. You can probably include Gmail as tested, as the test setup goes from the originating mail server out to Gmail and back to the main mail server.

So far, a loud Well Done!

And just to be otherwise, did you by any chance include any chr(0) trapping in there?

Rgds,

Chuck
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

chuckl wrote:It translates to a very emphatic - mail sent using 3.3.0 and sendmail would arrive blank - truncated after the header.
Switching to 3.3.1 it arrives perfect in all it's pristine glory. Suprised you didn't hear the loud whoops from Canada and West London in Manchester.

So far I've tested sendmail - server default (-t -i), sendmail Swift (-bs), smtpauth and smtp. You can probably include Gmail as tested, as the test setup goes from the originating mail server out to Gmail and back to the main mail server.

So far, a loud Well Done!

And just to be otherwise, did you by any chance include any chr(0) trapping in there?

Rgds,

Chuck
No check for NUL bytes. I'll add that now though. That problem is less likely to occur than the bug I just fixed. Basically the NUL byte problem could be put down to a bug in the text editor being used, whereas the <CRLF>.<CRLF> problem was 100% my fault (only affected QP encoding) :oops:
Post Reply