Page 1 of 1

How do I read/process very large mail files from PHP?

Posted: Tue Apr 10, 2007 10:32 am
by Kadanis
Basically, I'm stuck with this and looking for design pointers as much as code help... so any comments / suggestions / out right solutions :wink: will be appreciated

The website I'm currently working on sends out mailings to its opt-in lists. Originally all mailings came from post@domain.com, all the bounces returned to that box. It now has around 1.7 GB of mail in there (6000+ mails) some spam, some bounces.

I have been trying to write a mail processing script that will run off of CRON to access and process, then delete all these mails. Using something like

Code: Select all

$imap = imap_open("{".$server."/pop3:110/notls}INBOX", $user, $pass) or die (imap_last_error());
to handle the stream.

This works fine on the test mail box (with only 10 emails in it), but fails with the error

Code: Select all

POP3 connection broken as response
on the live mailbox.

To compound the issue, they recently changed their protcols so that now, all mailings come from listname@domain.com, so there are now 5 mailboxes to manage in this way.

All I need to do is get access to the mail box, use a preg to find a couple of ID's that should be in the headers (part of the outgoing mail) and then write them to a database, along with the bounce headers. If the code isn't found then its logged as an unknown and deleted.

Any suggestions?

Posted: Tue Apr 10, 2007 10:59 am
by Chris Corbyn
Probably not the answer you're looking for, but don't use those functions if you need to open very large files. I'm going to write an IMAP/POP library since I'm not aware of this functionality being present in other libraries but basically you need to filestream (i.e. copy to disk and copy back of disk, in small buffered chunks).

EDIT | I'm very sorry, I just replied to the title and not your actual question. If you're going to trawl through 1.7GB of emails with the IMAP functions I should point out another major flaw with them... they download full messages just to access the headers. You don't need to do that but they don't offer any other way. maybe you could try the IMAP stuff in PEAR?

Posted: Tue Apr 10, 2007 11:27 am
by Kadanis
Well thanks for looking anyway. It may be that you hit the nail on the head first time round.

The primary issue I have at the moment is the inability to even open the mailbox to read the contents, whether its a full message or just a header :(

I will have a look at the PEAR stuff to see if there is any thing I can use there. If I could just get the mailbox open it would be a start.