Page 1 of 2

Attachment Timeout

Posted: Wed May 02, 2007 2:41 pm
by madmike
I was using the example form("Sending an e-mail from a form with Swift") from the tutorial, but with sendmail instead of SMTP. I am able to send small jpg files, but large zips above 5mbs cause the script to timeout. I have increase post_max_limit, upload_max_filesize, and timeout numbers. While I was watching the "/tmp" folder in the server, I saw the temp file being cache. Once the temp file reach the same size as the attachement, another temp file appeared. It increase to the same file size, and a third temp file appeared. Then the script timedout, and the upload was canceled. Does anybody have an idea on what might be causing this?

Thx,
mike

Posted: Thu May 03, 2007 5:16 am
by Chris Corbyn
Swift's role in creating the tmp files would be over and done with in a split second so it's unlikely you'd ever actually see the filesize increasing without logging it somewhere every few microseconds. The various tmp files you are seeing are probably from other system processes.

Does this work if you change sendmail to SMTP?

By the way, it's possible to configure sendmail, exim and probably any other MTA to refuse attachments over a certain size, although this should cause an error rather than a script timeout. Have you modified the tutorial code in any other way to fit your needs better?

Posted: Thu May 03, 2007 8:53 am
by madmike
It is basically a windows box with xampp installed. The temp folder is used only by swift, since there is nothing else installed but wordpress. What I did was refresh the temp folder manually as the file was uploading. If I change the upload_file_size to 2mb, it gives me a file size exceeds error.

As for SMTP, can't use it. The way the box was setup, I have not been able to get an SMTP to run on it. I believe it is something with xampp, but thats another issue. Sendmail was the only way to get it working. Other then changing the two lines that reference SMTP to sendmail, I have not modified the code. If I have to use SMTP then I guess I am back to square one.

Mike

Posted: Thu May 03, 2007 9:01 am
by Chris Corbyn
Does your windows box have sendmail installed? I guess you have downloaded this and installed it manually?

Swift isn't actually uploading the file so if the error is occuring whilst the file is uploading then it's something related to PHP. Swift does create a few cache files whilst it composes the email however. It will create one for the attachment, one whilsts it encodes and one for the message body. Then it will glue those together while it sends.

Obviously something is going wrong after the upload, so we'll get to the bottom of why that is, but to narrow things down a bit, does the email actually end up in your inbox even though the script times out? (I'm wondering where the hang is occurring).

Posted: Thu May 03, 2007 9:42 am
by madmike
Ok, I think I found out the problem. My "max_execution_time" is too low for the size of files I am sending. It is set at 300. I think I am going to double that number and try again.

Error: Fatal error: Maximum execution time of 300 seconds exceeded in C:\Program Files\xampp\htdocs\swift\Swift\Cache\Disk.php on line 67

The form will be uploading 25MB files, and the error happens if the file is larger then 6MB. Oh well.

p.s. happy bday.

Posted: Thu May 03, 2007 9:54 am
by Chris Corbyn
I'm not 100%, but I don't think the actual time it takes to upload the file counts towards the execution time so post back if that doesn't solve it. I assume your emails are not being sent at all before the timeout error.
madmike wrote:p.s. happy bday
Thank you :) I'm off out to eat Spanish food tonight. Yummo!

Posted: Thu May 03, 2007 10:14 am
by madmike
Update: So my IT department finally answer my email from last month. hahaha. They gave me the SMTP server address. I switch back the code from sendmail to SMTP. From what I can tell the timeout is happening with swift putting the file together. It work by changing 300 to 600, but it timeouts when I try to upload a 20mb file. This is what I see happening when I left the max_time at 300 with a 9mb file.

I submit the form with a 9mb file, and in the "/tmp" directory I see the file being uploaded. Then I see a some temp files(random characters) appear. One of the temp files(ends with body) is increasing in size equal to the uploaded file. Once that is done the uploaded file disappears, and the temp file(with "body") is left. This happens real fast. A new file appears with the words "append". This slowly increments(takes about 5 minutes) in file size. Right when it gets near the same size as "body" file the script timesout, and all the temp files except the "body" one are gone. If i upload the a smaller file say around 6mbs, it does the same thing but succeeds in sending the email.

Could it just be the server is to slow in building the email?

Posted: Thu May 03, 2007 10:25 am
by Chris Corbyn
It's interesting hearing all that in action. Those files you described are indeed created by Swift when the Disk cache is used.

I'm a little confused about what's going on with the filesizes and times though. I'll have to dig into the code to refresh my memory on this but you should see something ending in "b64" which is larger than the attached file itself, something ending in "body" which is relatively small, something ending in "append" which is about the same size as the "b64" file and possibly a few others. Everything should happening lightning fast though so something is going a bit wonky there for you :(

Is the code exactly as it appears in the wiki or has it been modified to suit your needs? If it's been modified can you post the code please?

Posted: Thu May 03, 2007 10:48 am
by madmike
Yes, the b64 is the first file that I assume is the file being uploaded to the temp folder, and it is larger then the actual file. After "b64" is in the temp folder, "body" appears and starts to increase in size. Once it equals to "b64", "b64" disappears and "append" appears. You can see this in the screenshot below. The code is exactly the same as the one in the wiki. The only difference is the SMTP server address and my email address.

Image

I think it might be the whole xampp or maybe the way the network is here.

Posted: Thu May 03, 2007 11:31 am
by Chris Corbyn
Ah I didn't realise you were using PHP5 (I only make that assumptions as files disappear at runtime whilst objects are destructed rather than at the end of script execution). I'm just about to head out to eat but I'll ponder over this whilst I'm out ;)

If the temporary folder that's being written to is on a networked drive that will cause a big slowdown because Swift is processing your file (opening and closing the temp file) in chunks of just under 8 kilobytes. That's a lot of NFS requests for a 12MB file and you'll likely get better performance by increasing the "memory_limit" directive to 128MB (a stupidly safe value) and removing the call to "Swift_CacheFactory::setClassName()" and "Swift_Cache_Disk::setSavePath()" so that Swift doesn't try writing to your network drive.

Posted: Thu May 03, 2007 11:40 am
by madmike
Im off to lunch too. I'll try that out when I come back.

Posted: Thu May 03, 2007 11:44 am
by Chris Corbyn
madmike wrote:Im off to lunch too. I'll try that out when I come back.
I am of course assuming that /tmp points to a network drive here, but if it does, pointing it to a local filesystem instead will make this go as fast as it should do. You shouldn't really have to avoid the disk cache, and so far I've had good reports about it :)

Off out now... I'll check for updates later.

Posted: Thu May 03, 2007 2:16 pm
by madmike
so, your suggestion worked. Sending a 9mb file took only 30 seconds to process. Sending an 20Mb file took only 2 minutes. Let me see If I describe this right. There are three servers involved.

- form.html lives on file storage appliance
- handle_form.php and success.php live on windows box with php installed (application server)
- "/tmp" folder is also on the windows box(application server)
- SMTP server is its own box (mail server appliance) in a different building from the previous two boxes i think. :)

Since the boxes are all behind the same firewall, will the "/tmp" folder still process as slow as I describe?

Thanks again for all your help.
mike

p.s. I'm now going to modify the file to accepted multiple attachments. Woohoo!

Posted: Fri May 04, 2007 5:24 am
by Chris Corbyn
It sounds like /tmp would be accessed locally anyway (without going via the network) so I'm not sure. The sure way to find out would be to make a directory in the same place as handle_form.php, make it writable to PHP, then change the lines:

Code: Select all

Swift_CacheFactory::setClassName("Swift_Cache_Disk");
Swift_Cache_Disk::setSavePath("/tmp");

// to 

Swift_CacheFactory::setClassName("Swift_Cache_Disk");
Swift_Cache_Disk::setSavePath("./your_temp_dir");

Posted: Fri May 04, 2007 11:01 am
by madmike
That is the way it is setup.

handler_form.php
/tmp
/lib/Swift/

Should I move the "/tmp" into the "/lib" folder? Another question, Do you have a tutorial on multiple attachments? I can not get mine to work. If you don't have one, I'll open a new post with my code.