Page 1 of 1

Google News RSS problem

Posted: Wed Aug 24, 2005 12:28 pm
by funsutton
I am parsing the news from google just fine, but I notice that when I click some of the links, if it doesn't think I am from google, it gives me a registration page. So, I am triyng to figure out how to send a custom referer when a user clicks a news link. What I have so far is:

Code: Select all

function getlink($link)
{
$ch = curl_init("$link");
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_REFERER, "http://news.google.com");
        $data = curl_exec($ch);
        curl_close($ch); 
}


I added an onclick statement to the link to execute the getlink($link), however as soon as the page loads, it pulls up the article (not reg form, so we know that it is sending the referer) within my page (at the placement of the getlink() statement. I was hoping to open it in a new window or just not in my page.

Thanks,
Brian

Posted: Wed Aug 24, 2005 1:33 pm
by anjanesh
Strange - google news has external links - not google links. These pages can be viewed if you simply enter them in the browser.

I hope your link is something like this :
http://news.google.com/news?q=php+devel ... output=rss

Posted: Wed Aug 24, 2005 1:38 pm
by funsutton
anjanesh wrote:Strange - google news has external links - not google links. These pages can be viewed if you simply enter them in the browser.

I hope your link is something like this :
http://news.google.com/news?q=php+devel ... output=rss
No, you don't understand. If you click one of google's news links on their google news page, there are some websites that check the referer, and if it is google, it will let you read it. If you were to just copy and paste the link and your referer was not google, it wouldn't let you read it. This is only with some links...not all.

Thus, I am trying to send a custom referer as google so that all of the websites (that check the referer) will let me read the pages.

Thanks,
Brian

Posted: Wed Aug 24, 2005 2:03 pm
by anjanesh
Oh - I see - this is news !

What happens when you replace http://news.google.com with http://google.com or http://www.google.com ?

Posted: Wed Aug 24, 2005 2:06 pm
by funsutton
anjanesh wrote:Oh - I see - this is news !

What happens when you replace http://news.google.com with http://google.com or http://www.google.com ?
not sure, but I can get it working with news.google.com. I just can't get it to go to the page....instead it just reads the page in as input.

That's my problem. If I could just get curl to actually go to that page while posting the referer, it would be great.

-brian

Posted: Wed Aug 24, 2005 3:41 pm
by funsutton
NEVERMIND

I got it working by using google's redirect:

http://www.google.com/url?sa=D&q=$url

It's great...I don't even have to change the referer. While it would still be cool to know how to do that, I don't need it now.

Thanks for the responses.

Brian