parsing string obtained via cUrl

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
inosent1
Forum Commoner
Posts: 97
Joined: Wed Jan 28, 2009 12:18 pm

parsing string obtained via cUrl

Post by inosent1 »

i have this code

Code: Select all

<?php

function curl_download($Url){
     if (!function_exists('curl_init')){
        die('Sorry cURL is not installed!');
    }
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $Url);
    curl_setopt($ch, CURLOPT_REFERER, "http://www.mysite.org/ref.htm");
    curl_setopt($ch, CURLOPT_USERAGENT, "MozillaXYZ/1.0");
   curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, false);
    curl_setopt($ch, CURLOPT_TIMEOUT, 10);
    $output = curl_exec($ch);
     curl_close($ch);
 
    return $output;
}
$str = curl_download('http://mysite.com');
$mystring = $str;
$findme   = 'a';
$pos = strpos($mystring, $findme);

if ($pos === false) {
    echo "The string '$findme' was not found in the string '$mystring'";
} else {
    echo "The string '$findme' was found in the string '$mystring'";
    echo " and exists at position $pos";
}
?>
what i am trying to do is capture the source of the page and then locate a little bit of text at a particular spot in the source and then upload that data bit to a mysql DB via INSERT, etc

the above seems to be working but where you see my attempt to grab the source and turn it into a string variable obviously isnt working

for the next part of this, how can i grab the source, and turn that into a text string, so i can then properly use the strpos() thing?

any ideas are welcome, thanks!
greip
Forum Commoner
Posts: 39
Joined: Tue Aug 23, 2011 8:23 am
Location: Oslo, Norway

Re: parsing string obtained via cUrl

Post by greip »

Why not simply use file_get_contents()?

See the example on this page: http://php.net/manual/en/function.file-get-contents.php
phphelpme
Forum Contributor
Posts: 261
Joined: Sun Nov 21, 2010 3:32 pm

Re: parsing string obtained via cUrl

Post by phphelpme »

Hi there,

So you want to grab the page contents using cURL then save the contents into a string value so you can extract certain content using strpos() and substr() then save into a database.

I have helped about 5 different people on this forum in the last week with this so it would seem a confusing topic for some. :)

Anyway, I have on my blog the code you need to get it into a string: http://www.shaunellerton.co.uk/2011/08/ ... -php-curl/

But just for the sake of it I will post it here too and if you need anymore help then just pm me if needed:

Code: Select all

<?php
 
// sets url value
$url = "http://www.mysite.org/ref.htm";
 
// create curl resource
$ch = curl_init();
 
// set url
curl_setopt($ch, CURLOPT_URL, $url);
 
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 
// $output contains the output string
$output = curl_exec($ch);
 
// close curl resource to free up system resources
curl_close($ch);
 
//echo $output;
 
?>
The above code will grab the website page and load it all into a string variable called $output for you to do what ever you wish with it.

@greip cURL would be so much better but I understand why you suggested that. Good idea but I personally would advice using cURL but thats just me :)

Best wishes
inosent1
Forum Commoner
Posts: 97
Joined: Wed Jan 28, 2009 12:18 pm

Re: parsing string obtained via cUrl

Post by inosent1 »

thanks for the reply

here is what i have:

Code: Select all

<?php
 include("dbinfo.inc.php");
mysql_connect(localhost,$username,$password);
@mysql_select_db($database) or die( "Unable to select database");
// sets url value
$url = "http://inovacapitalmanagement.com/exa.php";
 
// create curl resource
$ch = curl_init();
 
// set url
curl_setopt($ch, CURLOPT_URL, $url);
 
//return the transfer as a string
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 
// $output contains the output string
$output = curl_exec($ch);
 
// close curl resource to free up system resources
curl_close($ch);


$findme   = 'US&nbsp;3';
$pos = strpos($output, $findme) + 77;
$dow = substr($output,$pos,5);
 echo substr($output,$pos,5); 
//echo $pos;
$query = "INSERT INTO data1 (dow) VALUES ('$dow')";
mysql_query($query);

mysql_close();

?>
it works great. i get the data and it uploads to the DB.

the next little hurdle is to have it updating every x time interval 1) without having to have the page open -- or if that is impossible, have it update the DB every x interval w/o a page refresh
phphelpme
Forum Contributor
Posts: 261
Joined: Sun Nov 21, 2010 3:32 pm

Re: parsing string obtained via cUrl

Post by phphelpme »

Setup cron Job on your host if it supports them to run a certain script say x amount of minutes apart.

Check out this site for more info: http://www.thesitewizard.com/general/set-cron-job.shtml

Best wishes
Post Reply