Page 1 of 1

Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 12:27 pm
by jboku
Okay, so I have a problem I can't seem to solve. It's driving me a little crazy and perhaps its something so small I am just over-looking it.

I am creating a small e-card system and I have a script that runs to remove people who want to opt-out of it. When someone adds up to 480 email addresses they want to send out, it works. If they add more than 480 it writes to the DB as blank.

Here is the code below:

index.php

Code: Select all

function post_variables($action,$variables){
            $ch = curl_init();
            curl_setopt($ch, CURLOPT_VERBOSE, 1);
            curl_setopt($ch, CURLOPT_URL, $action); //form action
            curl_setopt($ch, CURLOPT_FAILONERROR, 1); 
            curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
            curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
            curl_setopt($ch, CURLOPT_TIMEOUT, 3);
            curl_setopt($ch, CURLOPT_POST, 1);
            curl_setopt($ch, CURLOPT_POSTFIELDS,$variables);//form variables
            $result = curl_exec($ch); 
            curl_close($ch);
            return $result;
        }
 
$to_emails=post_variables("http://localhost/scripts/ecards/remove_opt_outs2.php","subject=Unclean Emails"."&email_list=".$to_emails."&sender_name=".$from_name."&sender_email=".$from_email."&email_subject=".$custom_greeting."&email_message=".$message."&image_name=".$images_web_location . $new_images[$id]."&image_url=".$target_url);
remove_opt_outs2.php

Code: Select all

//FUNCTION: GET TABLE DATA
function get_table_data($database_connection,$sql_query){
    //TABLE DATA
    //get all of the data from the table and record it to an array with the keys recording the primary key and the field attributes - field name|primary key value|field type|field length eg. title|13|text|128
    $all_data_array=array();
    $query_result=mysql_query($sql_query,$database_connection);
    while($query_row=mysql_fetch_row($query_result)){
        for($d=0;$d<sizeof($query_row);$d++){       
            //define the array
            if(!isset($all_data_array[mysql_field_name($query_result,$d)])){
                $all_data_array[mysql_field_name($query_result,$d)]=array();
            }
            //add the item's attribute to the attributes array
            $all_data_array[mysql_field_name($query_result,$d)][sizeof($all_data_array[mysql_field_name($query_result,$d)])]=$query_row[$d];            
            
        }
    }
    return $all_data_array;
}
 
function post_variables($action,$variables){
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_VERBOSE, 1);
    curl_setopt($ch, CURLOPT_URL, $action); //form action
    curl_setopt($ch, CURLOPT_FAILONERROR, 1); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_TIMEOUT, 3);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS,$variables);//form variables
    $result = curl_exec($ch); 
    curl_close($ch);
    return $result;
}
 
if(isset($_POST["email_list"])){
    $email_list=explode(",",$_POST["email_list"]);
    
    //Loop through every email address
    for($a=0;$a<sizeof($email_list);$a++){
    
        get_table_data($database_connection,"SELECT email_address FROM `opt_outs` WHERE email_address='".$email_list[$a]."'");
        //If this email address has been opted out, add it to the unclean list
        if(mysql_affected_rows()>0){
            $unclean_email_list[]=$email_list[$a];
        }
        //Otherwise add it to the clean list
        else{
            $clean_email_list[]=$email_list[$a];
        }
    }
//Record the ecard details
    if(isset($clean_email_list)){
        $included_recipients=sizeof($clean_email_list);
    }else{
        $included_recipients=0;
    }
    //Number of recipients who had previously opted out
    if(isset($unclean_email_list)){
        $excluded_recipients=sizeof($unclean_email_list);
    }else{
        $excluded_recipients=0;
    }
print implode(  ", ", $clean_email_list );
    
    //Email the ecard details
    post_variables("http://localhost/scripts/ecards/email_ecard_details.php",$ecard_info."&email_message=".$_POST["email_message"]."&recipient_email_addresses=".implode( ", ", $clean_email_list ));
    
}
One more thing that throws me for a loop, at the bottom of the opt_out page have it email me the information that is sent. In that email I see all the addresses added, even over 480, but when it writes to the DB it comes up blank. Even if I put echo $to_emails right after $to_emails=post_variables(... it still shows up blank if the email list exceeds 480 emails.

Can someone help point me in the right direction?

Thanks

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 1:48 pm
by JakeJ
My first thought, without actually looking at your code too closely is that you should echo out the results of an array with more than 480 addresses to see if something after that is corrupt or acting strange.

Is it always 480 exactly? Very strange.

If you're reaching some built in MySQL restraint, then the solution is to make a new array after 480 records.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 2:37 pm
by jboku
JakeJ wrote:My first thought, without actually looking at your code too closely is that you should echo out the results of an array with more than 480 addresses to see if something after that is corrupt or acting strange.

Is it always 480 exactly? Very strange.

If you're reaching some built in MySQL restraint, then the solution is to make a new array after 480 records.
It is kind of random. It was 480, now its 450ish.... Is it a time issue? I tried setting it to set_time_limit(0) but the same thing still :/. 480 is the max i've gotten. Now when I tried it it wont go over 450, nothing changed. I am VERY confused.... lol

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 4:03 pm
by JakeJ
did you echo out the results before the attempt to write to the database?

What happens if you write it to a text file?

Can you generate an array of a similar size of a different data type to see if that writes to the db using the same code?

To get to the bottom of it, you need to determine if it's a php or a mysql problem.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 5:12 pm
by Eran
Two things to check -
1. That you are not exceeding max_execution_time in your php.ini (you can set it using set_time_limit() )
2. That you are not running out of memory.

You should be working with display_errors enabled so you could see the error message that ends your script

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 5:26 pm
by JakeJ
I thought about the memory issue but didn't say anything because it shouldn't come up with less than 500 records. For that matter, the timing shouldn't either but anything is possible.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 5:34 pm
by Eran
it shouldn't come up with less than 500 records
This number is completely arbitrary. It depends on the size of the record, the memory allocation to PHP amongst other things. There is no rule that says that memory would only run out when you reach 500 records.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 5:52 pm
by JakeJ
pytrin wrote:
it shouldn't come up with less than 500 records
This number is completely arbitrary. It depends on the size of the record, the memory allocation to PHP amongst other things. There is no rule that says that memory would only run out when you reach 500 records.
Even so, an array of around 500 email address should not tax the memory of any but the very oldest servers.

Unless he's accidentally creating a new array each time an address is added to it. That would do it.

Joboku. Another piece of advice would be to manually create a PHP insert file with more than 500 addresses (not as hard as it sounds) and paste it in to phpmyadmin or whatever you're using to manage your database. That would help isolate the problem a bit.

ISOLATE, ISOLATE, ISOLATE. Like I said, don't assume the problem is PHP.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 5:59 pm
by Eran
Even so, an array of around 500 email address should not tax the memory of any but the very oldest servers.
it all depends on the memory allocation to PHP... the default is quite low.
The first order of business should be to turn on display_errors, he can't debug without knowing what the problem is.

Re: Results return blank if over 480 lines

Posted: Fri Feb 12, 2010 6:12 pm
by JakeJ
pytrin wrote:The first order of business should be to turn on display_errors, he can't debug without knowing what the problem is.
Agreed, but where's the fun in that? :banghead:

Re: Results return blank if over 480 lines

Posted: Mon Feb 15, 2010 10:20 am
by jboku
hey everyone,

Thank you all for your reply's. I am still having the same problem. Here is what I've done:

1 - Echo - I tried to echo it out but it shows up blank or as 1.
2 - I can write 500 or more emails to the DB no problem when I don't called the optout page / script listed above.
3 - display_error - I had a CURL warning and that's it. I fixed it but still have the same problem
- CURL warning: PHP Warning: curl_setopt() [<a href='function.curl-setopt'>function.curl-setopt</a>]: CURLOPT_FOLLOWLOCATION cannot be activated when in safe_mode or an open_basedir

To change that I replaced curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); with curl_setopt($ch, CURLOPT_HEADER, 0);

No more warnings / errors are being shown.

The strangest part to me is found at the end of the code.

Code: Select all

    //Return the clean list (with opted out email addresses removed)
    print implode( ",", $clean_email_list );
    
    //Email the ecard details
    post_variables("http://localhost/scripts/ecards/email_ecard_details.php",$ecard_info."&email_message=".$_POST["email_message"]."&recipient_email_addresses=".implode( ", ", $clean_email_list ));
When I call email_ecard_details.php it emails me all the details. That shows all 500 email addresses. So it's transferring there fine but not back to the page I submitted it on :/.

EDIT:

Also if I add another print implode( ",", $clean_email_list ); under the one up there and add 400 emails, it adds 799 to my DB no problem... why 799 and not 800? not sure :/. lol

Re: Results return blank if over 480 lines

Posted: Mon Feb 15, 2010 11:29 am
by jboku
Hey guys,

I fixed the problem and I feel stupid saying what it was :(.

curl_setopt($ch, CURLOPT_TIMEOUT, 3);

I changed that to a higher # (30) and it works now :/. I had to change it on the index.php page. I thought I did before but apparently I had only changed it on the optout page.

Thanks for all of your help. Sorry it was such a stupid problem lol.

Re: Results return blank if over 480 lines

Posted: Mon Feb 15, 2010 11:34 am
by JakeJ
Perhaps I've completely missed the boat, but why are you using cURL anyway if the page you're fetching is on your local host?