i am using curl to grab search results from web pages
i want to use it on this site
http://www.twitch.tv/search?query=dan+d ... dinh&stp=1
but the source code returned does not have the title, description etc in the source code
instead they have variables which look like {{title}}, {{description}} in its place
i have searched all of the java script files and all the other files in the source code for these variable values and cannot find them
but if the web browser can translate these variables into their values, there must be a way i can do it in php
does anyone know if there are other functions like curl that have the functionality i am looking for?
curl doesnt quite cut it
Moderator: General Moderators
- Christopher
- Site Administrator
- Posts: 13596
- Joined: Wed Aug 25, 2004 7:54 pm
- Location: New York, NY, US
Re: curl doesnt quite cut it
You will need to have a browser render the page after you fetch it. cURL just sends/gets requests -- it does not know how to render different data formats.
(#10850)
-
chopficaro
- Forum Commoner
- Posts: 68
- Joined: Fri Jan 01, 2010 12:56 am
Re: curl doesnt quite cut it
is there a way to simulate browser rendering?
Re: curl doesnt quite cut it
CasperJS is an awesome way to use a virtual browser to open a site, manipulate dom, take screenshots, click elements, submit forms, etc. On Ubuntu, you can install it with `sudo apt-get install casperjs` or OSX `brew install casperjs`. It uses a full V8 browser engine and executes JavaScript like a normal browser.chopficaro wrote:is there a way to simulate browser rendering?