Page 1 of 1
time out problem in curl
Posted: Sun Aug 03, 2008 7:01 am
by bhavin12300
hi
i am curl to fetch webpage from server.
i want to fetch two webpage one after another.
when i request one page from script,it is working perfectly.but when i insert code to fetch another page after first page it give me timeout error.
it worked perfectly fine if i request both page in individual script,but when i insert request code for file "two" after request code of file "one", it give me timeout error.
Code:
Code: Select all
$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?SEQ=1217741147969732008&breadcrumbStart=1", $referer="");
$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?SEQ=1217741147969732008&breadcrumbStart=2", $referer="");
These are two pages.
Again,
if i insert one of any request in script alone it is working just fine but not working if i put both together.
whats wrong in it???
Re: time out problem in curl
Posted: Sun Aug 03, 2008 7:36 am
by ody3307
This doesn't look like you are using cURL. You don't have to but you mentioned it in you post.
If you just want to grab these 2 pages, and do something with the results afterwards, you'll need to append the second page to the variable. As you have it written now, the second call overwrites the result of the first call.
So, if you want both pages in your variable, change $web_page = TO $web_page .= on the second call.
If you want to grab a page, do something with it, then grab another page, then put your code to "do something" between these 2 calls.
EDITED:
Oops forgot about changing timeout.
Use set_time_limit() to change the maximum time limit for the script.
Re: time out problem in curl
Posted: Sun Aug 03, 2008 11:48 am
by bhavin12300
sir,
http_get is my function as its definition is as follow.
Code: Select all
function http_get($target, $ref)
{
return http($target, $ref, $method="GET", $data_array="", EXCL_HEAD);
}
function http($target, $ref, $method, $data_array, $incl_head)
{
# Initialize PHP/CURL handle
$ch = curl_init();
# Prcess data, if presented
if(is_array($data_array))
{
# Convert data array into a query string (ie animal=dog&sport=baseball)
foreach ($data_array as $key => $value)
{
if(strlen(trim($value))>0)
$temp_string[] = $key . "=" . urlencode($value);
else
$temp_string[] = $key;
}
$query_string = join('&', $temp_string);
}
# HEAD method configuration
if($method == HEAD)
{
curl_setopt($ch, CURLOPT_HEADER, TRUE); // No http head
curl_setopt($ch, CURLOPT_NOBODY, TRUE); // Return body
}
else
{
# GET method configuration
if($method == GET)
{
if(isset($query_string))
$target = $target . "?" . $query_string;
curl_setopt ($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt ($ch, CURLOPT_POST, FALSE);
}
# POST method configuration
if($method == POST)
{
if(isset($query_string))
curl_setopt ($ch, CURLOPT_POSTFIELDS, $query_string);
curl_setopt ($ch, CURLOPT_POST, TRUE);
curl_setopt ($ch, CURLOPT_HTTPGET, FALSE);
}
curl_setopt($ch, CURLOPT_HEADER, $incl_head); // Include head as needed
curl_setopt($ch, CURLOPT_NOBODY, FALSE); // Return body
}
curl_setopt($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); // Cookie management.
curl_setopt($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT); // Timeout
curl_setopt($ch, CURLOPT_USERAGENT, WEBBOT_NAME); // Webbot name
curl_setopt($ch, CURLOPT_URL, $target); // Target site
curl_setopt($ch, CURLOPT_REFERER, $ref); // Referer value
curl_setopt($ch, CURLOPT_VERBOSE, FALSE); // Minimize logs
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt($ch, CURLOPT_MAXREDIRS, 4); // Limit redirections to four
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
# Create return array
$return_array['FILE'] = curl_exec($ch);
$return_array['STATUS'] = curl_getinfo($ch);
$return_array['ERROR'] = curl_error($ch);
# Close PHP/CURL handle
curl_close($ch);
# Return results
return $return_array;
}
and by the way i dont want to append one page with other .what i want to do is i want to grab first page and doing something on it and than want to grab second page.
but when i do that timer exceed.
Re: time out problem in curl
Posted: Sun Aug 03, 2008 1:22 pm
by ody3307
Okay, it looks like you are using Michael Shrenck's code library. I'm pretty familiar with it. I don't see anything obvious so far. Can we see the code where you request the first page, then the second page?
Thanks
Re: time out problem in curl
Posted: Mon Aug 04, 2008 4:46 am
by bhavin12300
Code: Select all
$web_page = http_get("http://travel.travelocity.com/flights/InitialSearch.do?dateLeavingTime=Anytime&dateReturningTime=Anytime&seniors=0&children=0&flightType=roundtrip&dateReturningMonth=Aug&dateReturningDay=23&adults=1&leavingFrom=PDX&dateLeavingMonth=Aug&goingTo=SJC&dateLeavingDay=20", $referer="");
$web_page = http_get("http://travel.travelocity.com/flights/ResolveAirportAction.do?".$seq, $referer="");
$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?".$seq."&breadcrumbStart=1", $referer="");
$web_page = http_get("http://travel.travelocity.com/flights/AirSearch.do?".$seq."&breadcrumbStart=2", $referer="");
echo $web_page['FILE'];
In these last two line i am getting time exceed error.if i remove any one from it than it worked perfectly nice.