How do I read/process very large mail files from PHP?

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
User avatar
Kadanis
Forum Contributor
Posts: 180
Joined: Tue Jun 20, 2006 8:55 am
Location: Dorset, UK
Contact:

How do I read/process very large mail files from PHP?

Post by Kadanis »

Basically, I'm stuck with this and looking for design pointers as much as code help... so any comments / suggestions / out right solutions :wink: will be appreciated

The website I'm currently working on sends out mailings to its opt-in lists. Originally all mailings came from post@domain.com, all the bounces returned to that box. It now has around 1.7 GB of mail in there (6000+ mails) some spam, some bounces.

I have been trying to write a mail processing script that will run off of CRON to access and process, then delete all these mails. Using something like

Code: Select all

$imap = imap_open("{".$server."/pop3:110/notls}INBOX", $user, $pass) or die (imap_last_error());
to handle the stream.

This works fine on the test mail box (with only 10 emails in it), but fails with the error

Code: Select all

POP3 connection broken as response
on the live mailbox.

To compound the issue, they recently changed their protcols so that now, all mailings come from listname@domain.com, so there are now 5 mailboxes to manage in this way.

All I need to do is get access to the mail box, use a preg to find a couple of ID's that should be in the headers (part of the outgoing mail) and then write them to a database, along with the bounce headers. If the code isn't found then its logged as an unknown and deleted.

Any suggestions?
User avatar
Chris Corbyn
Breakbeat Nuttzer
Posts: 13098
Joined: Wed Mar 24, 2004 7:57 am
Location: Melbourne, Australia

Post by Chris Corbyn »

Probably not the answer you're looking for, but don't use those functions if you need to open very large files. I'm going to write an IMAP/POP library since I'm not aware of this functionality being present in other libraries but basically you need to filestream (i.e. copy to disk and copy back of disk, in small buffered chunks).

EDIT | I'm very sorry, I just replied to the title and not your actual question. If you're going to trawl through 1.7GB of emails with the IMAP functions I should point out another major flaw with them... they download full messages just to access the headers. You don't need to do that but they don't offer any other way. maybe you could try the IMAP stuff in PEAR?
User avatar
Kadanis
Forum Contributor
Posts: 180
Joined: Tue Jun 20, 2006 8:55 am
Location: Dorset, UK
Contact:

Post by Kadanis »

Well thanks for looking anyway. It may be that you hit the nail on the head first time round.

The primary issue I have at the moment is the inability to even open the mailbox to read the contents, whether its a full message or just a header :(

I will have a look at the PEAR stuff to see if there is any thing I can use there. If I could just get the mailbox open it would be a start.
Post Reply