Page 1 of 1

No search results curl

Posted: Fri Jan 15, 2010 5:45 am
by anon404
I am trying to use curl to get html search results from the imdb website. The problem is when i echo the html result, there are no search results found for the given URL . When i enter the same URL i use for the getPage function in a browser, search results are found. Why isn't curl getting the imdb search results.

Code: Select all

 
    class crawler
    {
        var $ch;
        var $html;
        var $url;
 
        function crawler()
        {
            $this->search_html = "";
            $this->url = "";
        }
 
        function getPage($url)
        {
            $this->url = $url;
 
            if(isset($this->url))
            {
                $this->ch = curl_init();
                curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
                curl_setopt($this->ch, CURLOPT_URL, $this->url);
                $this->search_html = curl_exec($this->ch);
                curl_close($this->ch);
            }
 
        }
 
    }

Code: Select all

 
    $search_keywords = get_movies();
    $imdbCrawler = new crawler();
    
    foreach($search_keywords as $val)
    {
        $search = "http://www.imdb.com/find?s=all&q=";
        $search = $search.$val;
        $imdbCrawler->getPage($search);
        echo $imdbCrawler->search_html;
        break;
    }

Re: No search results curl

Posted: Fri Jan 15, 2010 11:21 am
by Sadam
When you say 'not getting them' do you mean it's returning the page successfully but there's just no matching results, or that it's failing and the page isn't being returned at all?

Re: No search results curl

Posted: Fri Jan 15, 2010 10:13 pm
by anon404
Page loads fine with no search results.

If i echo $string before line 9 in the crawler class, then paste the echoed string in a browser it successfully gets the page with the search results.

Edit: Solved