Page 1 of 1

parsing string obtained via cUrl

Posted: Wed Aug 24, 2011 12:21 am
by inosent1
i have this code

Code: Select all

<?php

function curl_download($Url){
     if (!function_exists('curl_init')){
        die('Sorry cURL is not installed!');
    }
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $Url);
    curl_setopt($ch, CURLOPT_REFERER, "http://www.mysite.org/ref.htm");
    curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");
   curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    $output = curl_exec($ch);
     curl_close($ch);
 
    return $output;
}
$str = curl_download('http://mysite.com');
$mystring = $str;
$findme   = 'a';
$pos = strpos($mystring, $findme);

if ($pos === false) {
    echo "The string '$findme' was not found in the string '$mystring'";
} else {
    echo "The string '$findme' was found in the string '$mystring'";
    echo " and exists at position $pos";
}
?>
what i am trying to do is capture the source of the page and then locate a little bit of text at a particular spot in the source and then upload that data bit to a mysql DB via INSERT, etc

the above seems to be working but where you see my attempt to grab the source and turn it into a string variable obviously isnt working

for the next part of this, how can i grab the source, and turn that into a text string, so i can then properly use the strpos() thing?

any ideas are welcome, thanks!

Re: parsing string obtained via cUrl

Posted: Wed Aug 24, 2011 5:30 am
by greip
Why not simply use file_get_contents()?

See the example on this page: http://php.net/manual/en/function.file-get-contents.php

Re: parsing string obtained via cUrl

Posted: Wed Aug 24, 2011 5:59 am
by phphelpme
Hi there,

So you want to grab the page contents using cURL then save the contents into a string value so you can extract certain content using strpos() and substr() then save into a database.

I have helped about 5 different people on this forum in the last week with this so it would seem a confusing topic for some. :)

Anyway, I have on my blog the code you need to get it into a string: http://www.shaunellerton.co.uk/2011/08/ ... -php-curl/

But just for the sake of it I will post it here too and if you need anymore help then just pm me if needed:

Code: Select all

<?php
 
// sets url value
$url = "http://www.mysite.org/ref.htm";
 
// create curl resource
$ch = curl_init();
 
// set url
curl_setopt($ch, CURLOPT_URL, $url);
 
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 
// $output contains the output string
$output = curl_exec($ch);
 
// close curl resource to free up system resources
curl_close($ch);
 
//echo $output;
 
?>
The above code will grab the website page and load it all into a string variable called $output for you to do what ever you wish with it.

@greip cURL would be so much better but I understand why you suggested that. Good idea but I personally would advice using cURL but thats just me :)

Best wishes

Re: parsing string obtained via cUrl

Posted: Wed Aug 24, 2011 8:02 pm
by inosent1
thanks for the reply

here is what i have:

Code: Select all

<?php
 include("dbinfo.inc.php");
mysql_connect(localhost,$username,$password);
@mysql_select_db($database) or die( "Unable to select database");
// sets url value
$url = "http://inovacapitalmanagement.com/exa.php";
 
// create curl resource
$ch = curl_init();
 
// set url
curl_setopt($ch, CURLOPT_URL, $url);
 
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 
// $output contains the output string
$output = curl_exec($ch);
 
// close curl resource to free up system resources
curl_close($ch);


$findme   = 'US&nbsp;3';
$pos = strpos($output, $findme) + 77;
$dow = substr($output,$pos,5);
 echo substr($output,$pos,5); 
//echo $pos;
$query = "INSERT INTO data1 (dow) VALUES ('$dow')";
mysql_query($query);

mysql_close();

?>
it works great. i get the data and it uploads to the DB.

the next little hurdle is to have it updating every x time interval 1) without having to have the page open -- or if that is impossible, have it update the DB every x interval w/o a page refresh

Re: parsing string obtained via cUrl

Posted: Thu Aug 25, 2011 2:36 am
by phphelpme
Setup cron Job on your host if it supports them to run a certain script say x amount of minutes apart.

Check out this site for more info: http://www.thesitewizard.com/general/set-cron-job.shtml

Best wishes