time out problem in curl

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
bhavin12300
Forum Newbie
Posts: 3
Joined: Fri Jan 26, 2007 11:00 pm

time out problem in curl

Post by bhavin12300 »

hi
i am curl to fetch webpage from server.
i want to fetch two webpage one after another.

when i request one page from script,it is working perfectly.but when i insert code to fetch another page after first page it give me timeout error.
it worked perfectly fine if i request both page in individual script,but when i insert request code for file "two" after request code of file "one", it give me timeout error.
Code:

Code: Select all

$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?SEQ=1217741147969732008&breadcrumbStart=1", $referer="");
$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?SEQ=1217741147969732008&breadcrumbStart=2", $referer="");
These are two pages.
Again,
if i insert one of any request in script alone it is working just fine but not working if i put both together.
whats wrong in it???
ody3307
Forum Newbie
Posts: 21
Joined: Wed Jul 30, 2008 7:29 am

Re: time out problem in curl

Post by ody3307 »

This doesn't look like you are using cURL. You don't have to but you mentioned it in you post.

If you just want to grab these 2 pages, and do something with the results afterwards, you'll need to append the second page to the variable. As you have it written now, the second call overwrites the result of the first call.

So, if you want both pages in your variable, change $web_page = TO $web_page .= on the second call.

If you want to grab a page, do something with it, then grab another page, then put your code to "do something" between these 2 calls.

EDITED:
Oops forgot about changing timeout.

Use set_time_limit() to change the maximum time limit for the script.
bhavin12300
Forum Newbie
Posts: 3
Joined: Fri Jan 26, 2007 11:00 pm

Re: time out problem in curl

Post by bhavin12300 »

sir,
http_get is my function as its definition is as follow.

Code: Select all

function http_get($target, $ref)
    {
    return http($target, $ref, $method="GET", $data_array="", EXCL_HEAD);
    }
function http($target, $ref, $method, $data_array, $incl_head)
    {
    # Initialize PHP/CURL handle
    $ch = curl_init();
 
    # Prcess data, if presented
    if(is_array($data_array))
        {
        # Convert data array into a query string (ie animal=dog&sport=baseball)
        foreach ($data_array as $key => $value) 
            {
            if(strlen(trim($value))>0)
                $temp_string[] = $key . "=" . urlencode($value);
            else
                $temp_string[] = $key;
            }
        $query_string = join('&', $temp_string);
        }
        
    # HEAD method configuration
    if($method == HEAD)
        {
        curl_setopt($ch, CURLOPT_HEADER, TRUE);                // No http head
        curl_setopt($ch, CURLOPT_NOBODY, TRUE);                // Return body
        }
    else
        {
        # GET method configuration
        if($method == GET)
            {
            if(isset($query_string))
                $target = $target . "?" . $query_string;
            curl_setopt ($ch, CURLOPT_HTTPGET, TRUE); 
            curl_setopt ($ch, CURLOPT_POST, FALSE); 
            }
        # POST method configuration
        if($method == POST)
            {
            if(isset($query_string))
                curl_setopt ($ch, CURLOPT_POSTFIELDS, $query_string);
            curl_setopt ($ch, CURLOPT_POST, TRUE); 
            curl_setopt ($ch, CURLOPT_HTTPGET, FALSE); 
            }
        curl_setopt($ch, CURLOPT_HEADER, $incl_head);   // Include head as needed
        curl_setopt($ch, CURLOPT_NOBODY, FALSE);        // Return body
        }
        
    curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);   // Cookie management.
    curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
    curl_setopt($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT);    // Timeout
    curl_setopt($ch, CURLOPT_USERAGENT, WEBBOT_NAME);   // Webbot name
    curl_setopt($ch, CURLOPT_URL, $target);             // Target site
    curl_setopt($ch, CURLOPT_REFERER, $ref);            // Referer value
    curl_setopt($ch, CURLOPT_VERBOSE, FALSE);           // Minimize logs
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);    // No certificate
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);     // Follow redirects
    curl_setopt($ch, CURLOPT_MAXREDIRS, 4);             // Limit redirections to four
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);     // Return in string
    
    # Create return array
    $return_array['FILE']   = curl_exec($ch); 
    $return_array['STATUS'] = curl_getinfo($ch);
    $return_array['ERROR']  = curl_error($ch);
    
    # Close PHP/CURL handle
    curl_close($ch);
    
    # Return results
    return $return_array;
    }
 
 
and by the way i dont want to append one page with other .what i want to do is i want to grab first page and doing something on it and than want to grab second page.
but when i do that timer exceed.
ody3307
Forum Newbie
Posts: 21
Joined: Wed Jul 30, 2008 7:29 am

Re: time out problem in curl

Post by ody3307 »

Okay, it looks like you are using Michael Shrenck's code library. I'm pretty familiar with it. I don't see anything obvious so far. Can we see the code where you request the first page, then the second page?

Thanks
bhavin12300
Forum Newbie
Posts: 3
Joined: Fri Jan 26, 2007 11:00 pm

Re: time out problem in curl

Post by bhavin12300 »

Code: Select all

 
$web_page = http_get("http://travel.travelocity.com/flights/InitialSearch.do?dateLeavingTime=Anytime&dateReturningTime=Anytime&seniors=0&children=0&flightType=roundtrip&dateReturningMonth=Aug&dateReturningDay=23&adults=1&leavingFrom=PDX&dateLeavingMonth=Aug&goingTo=SJC&dateLeavingDay=20", $referer="");
 
  
   $web_page = http_get("http://travel.travelocity.com/flights/ResolveAirportAction.do?".$seq, $referer="");
  
 
 
 
 $web_page  = http_get("http://travel.travelocity.com/flights/AirSearch.do?".$seq."&breadcrumbStart=1", $referer="");
 $web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?".$seq."&breadcrumbStart=2", $referer="");
echo $web_page['FILE'];
In these last two line i am getting time exceed error.if i remove any one from it than it worked perfectly nice.
Post Reply