Can Curl be blocked?

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
Rox23
Forum Newbie
Posts: 3
Joined: Tue Jul 27, 2010 8:41 am

Can Curl be blocked?

Post by Rox23 »

I was wondering if there was a way to block Curl POST requests?

I have a website that I am trying to use Curl with and no matter what I try, the page does NOT recognize the POST fields. The website was originally done by someone else. I have gone through the code and don't see anything that could do this?

I tried pointing the Curl POST code to another server and it all works. So my belief is there must be something blocking the curl request.

Any ideas?

Thanks!
User avatar
Benjamin
Site Administrator
Posts: 6935
Joined: Sun May 19, 2002 10:24 pm

Re: Can Curl be blocked?

Post by Benjamin »

Maybe the site is sending a header redirect because there are no cookies set, or you are not logged in.
Rox23
Forum Newbie
Posts: 3
Joined: Tue Jul 27, 2010 8:41 am

Re: Can Curl be blocked?

Post by Rox23 »

Benjamin wrote:Maybe the site is sending a header redirect because there are no cookies set, or you are not logged in.
I thought of that and I set cookies in the curl script.

I have tried using several diferent user agents and tried using "1" instead of "true". The results are always the same.

I have tried posting the fields as an array and as a concatenated string - no change.

Code: Select all

$target_url = $webs['website_url'];
$userAgent = 'Googlebot/2.1 (http://www.googlebot.com/bot.html)';
$fields = array();
$fields_string = "";

$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT,$_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_URL,$target_url);
foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; }
$fields_string = rtrim($fields_string,'&');
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$fields_string); 
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_HEADER      ,false );  
curl_setopt($ch, CURLOPT_TIMEOUT, 1000);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
$html= curl_exec($ch);
Last edited by Benjamin on Tue Jul 27, 2010 10:04 am, edited 1 time in total.
Reason: Added [syntax=php] tags.
User avatar
Benjamin
Site Administrator
Posts: 6935
Joined: Sun May 19, 2002 10:24 pm

Re: Can Curl be blocked?

Post by Benjamin »

Well, google doesn't send POST requests, so I'm not sure why you would be using that user agent. Are you sure this is your site? Have you checked to see what is being returned?
Rox23
Forum Newbie
Posts: 3
Joined: Tue Jul 27, 2010 8:41 am

Re: Can Curl be blocked?

Post by Rox23 »

I found that user agent code online somewhere.

I am just trying anything that could make sense to work. I have tried browser user agents (mozilla, netscape, etc...) and nothing.

I tried removing all the cURL options except the POST stuff and URL. Still won't work.

I get no post field results. It is as if the POST never happened.

Is there code that can block this type of POST that I need to look for in this website or server configuration?

Thanks!
Post Reply