Help with curl

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
ngmix
Forum Newbie
Posts: 4
Joined: Sun Oct 12, 2008 10:59 pm

Help with curl

Post by ngmix »

Hello all, I have a yahoogroups message board.
For effectiveness, I need to move some of the messages to my private board on my website.

When I use Curl to download the message, I get this phrase:
Aren't you supposed to be somewhere else?

Has anyone done this before,
please assist! how can I get it done.

Thank you.
User avatar
onion2k
Jedi Mod
Posts: 5263
Joined: Tue Dec 21, 2004 5:03 pm
Location: usrlab.com

Re: Help with curl

Post by onion2k »

By the sounds of it Yahoo's site is expecting the correct cookie/session/referer/a combination of all three, and you're not sending it. Basically you'll need to write something that follows the entire process from logging in to fetching the message as a browser would, but in PHP.
ngmix
Forum Newbie
Posts: 4
Joined: Sun Oct 12, 2008 10:59 pm

Re: Help with curl

Post by ngmix »

thanks for the response, this is my code here:

// I have created a file for cookiefile, one for cookiejar, and one for sample.txt and they are all world readable and writeable
//after all check has been done, download the new id's file - Im going to use curl.
$file = "sample.txt";
$url = "http://groups.yahoo.com/group/NaijaPoli ... age/138888";
echo cURLdownload( $url, $file);

function cURLdownload($url, $file)
{
//if( !cURLcheckBasicFunctions() ) return "UNAVAILABLE: cURL Basic Functions";
$ch = curl_init();
if($ch)
{
$fp = fopen($file, "w");
if($fp)
{
if( !curl_setopt($ch, CURLOPT_URL, $url) ) return "FAIL: curl_setopt(CURLOPT_URL)";
if( !curl_setopt($ch, CURLOPT_FILE, $fp) ) return "FAIL: curl_setopt(CURLOPT_FILE)";
if( !curl_setopt($ch, CURLOPT_HEADER, 0) ) return "FAIL: curl_setopt(CURLOPT_HEADER)";
if( !curl_setopt($ch, CURLOPT_COOKIEFILE, './cukiefile') ) return "FAIL: curl_setopt(CURLOPT_cookiefile)";
if( !curl_setopt($ch, CURLOPT_COOKIEJAR, './cukiejar') ) return "FAIL: curl_setopt(CURLOPT_cookiejar)";
if( !curl_exec($ch) ) return "FAIL: curl_exec()";
curl_close($ch);
fclose($fp);
return "SUCCESS: $file [$url]";
}
else return "FAIL: fopen()";
}
else return "FAIL: curl_init()";
}

After all this, I get a response in my sample.txt that says Aren't you supposed to be somewhere else?
I know that I am doing something wrong with the cookie/session setting, I just don't know what it is.
I'ld appreciate it if anyone can help.

Thanks again.
User avatar
onion2k
Jedi Mod
Posts: 5263
Joined: Tue Dec 21, 2004 5:03 pm
Location: usrlab.com

Re: Help with curl

Post by onion2k »

If I go to that Yahoo page it redirects me to the login page. I think that's pretty much certain that you'll need to make your PHP script log into Yahoo before it can access the right pages. You'll need to examine the login form to figure out what to send...
ngmix
Forum Newbie
Posts: 4
Joined: Sun Oct 12, 2008 10:59 pm

Re: Help with curl

Post by ngmix »

Thanks again,

but whenever I try running the program, my machine is already logged into yahoo where I am already a registered user of the forum. When I type the address into my address bar, I am able to see the page. Just doesn't work when I try to download using the program.
User avatar
onion2k
Jedi Mod
Posts: 5263
Joined: Tue Dec 21, 2004 5:03 pm
Location: usrlab.com

Re: Help with curl

Post by onion2k »

ngmix wrote:but whenever I try running the program, my machine is already logged into yahoo where I am already a registered user of the forum. When I type the address into my address bar, I am able to see the page. Just doesn't work when I try to download using the program.
Whichever browser you've used is logged into yahoo. Your machine isn't. For example, if you log into Yahoo with Firefox, then look at the page with IE7, IE7 won't be logged in. It's because each session is unique to each browser.

PHP with Curl is essentially another browser in that respect. There is no way to cheat*. You'll have to write the code to log into Yahoo with PHP. It's going to be a lot of work.

* Actually there is. You could copy the value of the Yahoo cookies when you log in using an ordinary browser into Curl's cookiejar file in PHP. It'd be a little unreliable though, and Yahoo probably check the user agent to block that anyway.
ngmix
Forum Newbie
Posts: 4
Joined: Sun Oct 12, 2008 10:59 pm

Re: Help with curl

Post by ngmix »

onion2k wrote:
* Actually there is. You could copy the value of the Yahoo cookies when you log in using an ordinary browser into Curl's cookiejar file in PHP. It'd be a little unreliable though, and Yahoo probably check the user agent to block that anyway.
I tried copying the cookie into a textfile, I used cookiejar and cookiefile as shown in the code, to store the cookies. I think yahoo blocked that already. I appreciate you helping me along.
How can I write my own code to log into yahoo. If I can get a guide, I don't mind spending the time to write it. I'm well versed with php, if you can just point me in the right direction.

Thanks a lot.
Post Reply