Google News RSS problem

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
funsutton
Forum Newbie
Posts: 9
Joined: Sun Jan 18, 2004 12:02 pm

Google News RSS problem

Post by funsutton »

I am parsing the news from google just fine, but I notice that when I click some of the links, if it doesn't think I am from google, it gives me a registration page. So, I am triyng to figure out how to send a custom referer when a user clicks a news link. What I have so far is:

Code: Select all

function getlink($link)
{
$ch = curl_init("$link");
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_REFERER, "http://news.google.com");
        $data = curl_exec($ch);
        curl_close($ch); 
}


I added an onclick statement to the link to execute the getlink($link), however as soon as the page loads, it pulls up the article (not reg form, so we know that it is sending the referer) within my page (at the placement of the getlink() statement. I was hoping to open it in a new window or just not in my page.

Thanks,
Brian
User avatar
anjanesh
DevNet Resident
Posts: 1679
Joined: Sat Dec 06, 2003 9:52 pm
Location: Mumbai, India

Post by anjanesh »

Strange - google news has external links - not google links. These pages can be viewed if you simply enter them in the browser.

I hope your link is something like this :
http://news.google.com/news?q=php+devel ... output=rss
funsutton
Forum Newbie
Posts: 9
Joined: Sun Jan 18, 2004 12:02 pm

Post by funsutton »

anjanesh wrote:Strange - google news has external links - not google links. These pages can be viewed if you simply enter them in the browser.

I hope your link is something like this :
http://news.google.com/news?q=php+devel ... output=rss
No, you don't understand. If you click one of google's news links on their google news page, there are some websites that check the referer, and if it is google, it will let you read it. If you were to just copy and paste the link and your referer was not google, it wouldn't let you read it. This is only with some links...not all.

Thus, I am trying to send a custom referer as google so that all of the websites (that check the referer) will let me read the pages.

Thanks,
Brian
User avatar
anjanesh
DevNet Resident
Posts: 1679
Joined: Sat Dec 06, 2003 9:52 pm
Location: Mumbai, India

Post by anjanesh »

Oh - I see - this is news !

What happens when you replace http://news.google.com with http://google.com or http://www.google.com ?
funsutton
Forum Newbie
Posts: 9
Joined: Sun Jan 18, 2004 12:02 pm

Post by funsutton »

anjanesh wrote:Oh - I see - this is news !

What happens when you replace http://news.google.com with http://google.com or http://www.google.com ?
not sure, but I can get it working with news.google.com. I just can't get it to go to the page....instead it just reads the page in as input.

That's my problem. If I could just get curl to actually go to that page while posting the referer, it would be great.

-brian
funsutton
Forum Newbie
Posts: 9
Joined: Sun Jan 18, 2004 12:02 pm

Post by funsutton »

NEVERMIND

I got it working by using google's redirect:

http://www.google.com/url?sa=D&q=$url

It's great...I don't even have to change the referer. While it would still be cool to know how to do that, I don't need it now.

Thanks for the responses.

Brian
Post Reply