Sending a lot of e-mail (mailinglist-like)
Moderator: General Moderators
Sending a lot of e-mail (mailinglist-like)
This is what the client asks:
There are groups of people, each with an e-mail address. These can be mailed all together, or only a few at a time.
The total number of people will probably exceed 1000-1500. The messages that have to be sent are personal, so each one has a more or less personal subject or content (template-wise, replacing "Dear %NAME%" with the user's name for example).
This has to be realtime. The only technologies that I can use are PHP and MySQL. To send a 1000 messages takes up a lot of time, and the page that will be sending these messages will probably timeout before it reaches the end, so I came up with an, in my opinion, pretty clean solution, and a pretty crappy one too. I tried to make my own version of threading in PHP, since that is not possible.
I would like to present the user with a progress bar too. Staring at a blank page is pretty annoying.
1. Using AJAX
I create a JavaScript array with all the addresses, names, personal data I need in it, and loop over it, using xmlhttprequest to call a page called ie. mail.php. I can pass parameters etc and all mail.php will have to do is send one message.
This might work fine, but when I let the object terminate its task before the next iteration, the user's browser just hangs untill the last iteration, causing my progressbar to jump from 0 to 100%.
If I don't wait untill the object has finished before goin into the next iteration, it all goes too fast and not all messages are sent.
2. Using the image preloading in PHP
Since the solution above did not seem to work, I made a simple php file which returns a 1px gif, and let it preload 1000 times for example, each time called with a different parameter.
Problem here is browser caching. Some browsers seem to ignore my Cache-control header, and some messages are, again, not processed.
Now I know that all of this is a very bad idea, and that I should use specific software etc, but the client really wants it this way. I tried convincing him but... you all know the situation I think.
Can anyone help me out here? Either with a way to make AJAX useable in this case (altough it is kind of abusing this nice 'technology'), or with a better solution to my problem?
All help is really appreciated!
There are groups of people, each with an e-mail address. These can be mailed all together, or only a few at a time.
The total number of people will probably exceed 1000-1500. The messages that have to be sent are personal, so each one has a more or less personal subject or content (template-wise, replacing "Dear %NAME%" with the user's name for example).
This has to be realtime. The only technologies that I can use are PHP and MySQL. To send a 1000 messages takes up a lot of time, and the page that will be sending these messages will probably timeout before it reaches the end, so I came up with an, in my opinion, pretty clean solution, and a pretty crappy one too. I tried to make my own version of threading in PHP, since that is not possible.
I would like to present the user with a progress bar too. Staring at a blank page is pretty annoying.
1. Using AJAX
I create a JavaScript array with all the addresses, names, personal data I need in it, and loop over it, using xmlhttprequest to call a page called ie. mail.php. I can pass parameters etc and all mail.php will have to do is send one message.
This might work fine, but when I let the object terminate its task before the next iteration, the user's browser just hangs untill the last iteration, causing my progressbar to jump from 0 to 100%.
If I don't wait untill the object has finished before goin into the next iteration, it all goes too fast and not all messages are sent.
2. Using the image preloading in PHP
Since the solution above did not seem to work, I made a simple php file which returns a 1px gif, and let it preload 1000 times for example, each time called with a different parameter.
Problem here is browser caching. Some browsers seem to ignore my Cache-control header, and some messages are, again, not processed.
Now I know that all of this is a very bad idea, and that I should use specific software etc, but the client really wants it this way. I tried convincing him but... you all know the situation I think.
Can anyone help me out here? Either with a way to make AJAX useable in this case (altough it is kind of abusing this nice 'technology'), or with a better solution to my problem?
All help is really appreciated!
- feyd
- Neighborhood Spidermoddy
- Posts: 31559
- Joined: Mon Mar 29, 2004 3:24 pm
- Location: Bothell, Washington, USA
it'd be fairly simple to create a launch script that, after checking if it can start such a mass-mailing and verifying access privledges to the script itself, would simply queue all the emails (in a table). Using a cron that runs every 5 minutes or something that checks to see if a queue is available, then selects a group out and emails them. Each email sent, it should remove the person from the queue.
The cron should update a second table with how much it's done, when it started, when it finished.. A third table could store any errors recieved along with data about the record involved. Some servers have a limit of how many emails can be sent within a specific period of time, so you will need to adjust the amount the cron grabs by that much.
at any rate, we have talked about this several times in the past. Search for "mass mail"
The cron should update a second table with how much it's done, when it started, when it finished.. A third table could store any errors recieved along with data about the record involved. Some servers have a limit of how many emails can be sent within a specific period of time, so you will need to adjust the amount the cron grabs by that much.
at any rate, we have talked about this several times in the past. Search for "mass mail"
First of all, thank you for your reply!
I have been thinking about that before talking to my client, and using cron (or the like) is not acceptable to him. He wants it realtime, as in: I click the button and the emails should go out.
I will retry to convince him, I know that your solution is the better one, but if his answer is still no I still will have to look for another answer. That's what I'm trying to find out now...
And oh yes, I searched the forums, but nothing suitable came up at first sight, that is why I asked my question. I will go throug the threads more in depth, maybe I can find another idea...
I have been thinking about that before talking to my client, and using cron (or the like) is not acceptable to him. He wants it realtime, as in: I click the button and the emails should go out.
I will retry to convince him, I know that your solution is the better one, but if his answer is still no I still will have to look for another answer. That's what I'm trying to find out now...
And oh yes, I searched the forums, but nothing suitable came up at first sight, that is why I asked my question. I will go throug the threads more in depth, maybe I can find another idea...
- feyd
- Neighborhood Spidermoddy
- Posts: 31559
- Joined: Mon Mar 29, 2004 3:24 pm
- Location: Bothell, Washington, USA
they would start nearly immediately....
a second option is to use an http-multipart, but your server must support flush(). Basically, you have a script that initiates the transaction (starts the sending script via a http subquery with a quick timeout. Your launching script then polls on a regular basis (not all the time, because you'll inhale the processor) the table containing the remaining addresses to send to. This will allow it to display an "active" status, yet not disturb the mailing script.
You can set the mailing script to not time out, however that partly depends on your server allowing such a setting.
http://www.phpclasses.org/browse/package/1704.html is a helpful tool for multipart output, you can find more of how it works on that page.
a second option is to use an http-multipart, but your server must support flush(). Basically, you have a script that initiates the transaction (starts the sending script via a http subquery with a quick timeout. Your launching script then polls on a regular basis (not all the time, because you'll inhale the processor) the table containing the remaining addresses to send to. This will allow it to display an "active" status, yet not disturb the mailing script.
You can set the mailing script to not time out, however that partly depends on your server allowing such a setting.
http://www.phpclasses.org/browse/package/1704.html is a helpful tool for multipart output, you can find more of how it works on that page.
I will investigate that further, might even be a better solution, thanks!
In the meanwhile someone gave me this piece of code, and wonderfully, it seems to work!
No timeout, adapted it a little, added a primitive progressbar and tried it with a 1000 iterations (including the sleep(1); )
In the meanwhile someone gave me this piece of code, and wonderfully, it seems to work!
No timeout, adapted it a little, added a primitive progressbar and tried it with a 1000 iterations (including the sleep(1); )
Code: Select all
<?php
$usermails[0] = "Ndsdqsjd Skqjsd";
$usermails[1] = "Aze Rty";
$usermails[2] = "Pom Bleh";
$usermails[3] = "Ajsld Sqsdk";
$usermails[3] = "Qqsdlk Qqsd";
?>
Sending mail to <span id="show_mail_info">Loading...</span>
<div style="width: 305px; border: solid black 1px;">
<div id="progress" style="background-color: blue; border: solid blue 10px;"></div>
</div>
<?
if (ob_get_level() == 0) ob_start();
for($i=0; $i<4; $i++)
{
$user = $usermails[$i];
?><script type="text/javascript">document.getElementById('progress').style.width = <?=($i * 95)?>;document.getElementById('show_mail_info').innerHTML="<?=$user[1]?>";</script><?php
ob_flush();
flush();
sleep(1);
}
ob_end_flush();
?>Start a session.
send emails 1-10.
save that info in the session array.
then do header('Location: http://'.$_SERVER['HTTP_HOST'].'/'.$_SERVER['PHP_SELF']);
That will reset the timer.
Now read the sessions array to see where you are: I already sent 1-10 so now I will send 11-20
Keep doing this until you have sent 1500.
send emails 1-10.
save that info in the session array.
then do header('Location: http://'.$_SERVER['HTTP_HOST'].'/'.$_SERVER['PHP_SELF']);
That will reset the timer.
Now read the sessions array to see where you are: I already sent 1-10 so now I will send 11-20
Keep doing this until you have sent 1500.