PHP Developers Network

A community of PHP developers offering assistance, advice, discussion, and friendship.
 
Loading
It is currently Thu Oct 19, 2017 5:47 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: cURL Experiments
PostPosted: Tue May 23, 2017 5:39 pm 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
Ladies & Gentlemen!

Oops! Let me try again:
Gentle Ladies & Hard Men (after-all, it's the ladies who are gentle compared to men and men hard, rough & tough compared to the ladies)!
And no, don't jest by saying "hard men" sounds like men having an erec**ion as some foolish men joked on other forums (busines/internet marketing) few months (or was it a yr back ?) back! :lol:
Get used to me addressing you as "hard men" because that's how I'm gonna do it frequently. :rofl:

Anyway, this thread is about cURL.
I will try building a unique script based on cURL. But, let's get rolling to learn first!
I have some viral traffic & viral money earning ideas and cURL will impliment them. Stick around and see how deep the rabbit hole is and what comes out of it! (Not joking!).


Last edited by UniqueIdeaMan on Tue May 23, 2017 6:25 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Tue May 23, 2017 5:42 pm 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
cURL Sample 1:


Why do you reckon the following code sample is not showing any page loading ? I see a complete blank page loading on my xampp.

Code:
<?php

//This code was found on: http://www.binarytides.com/php-curl-tutorial-beginners/
//gets the data from a URL
function get_url($url)
{
    $ch = curl_init();
     
    if($ch === false)
    {
        die('Failed to create curl object');
    }
     
    $timeout = 5;
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
    $data = curl_exec($ch);
    curl_close($ch);
    return $data;
}

echo get_url('http://www.apple.com/');
?>



Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Tue May 23, 2017 5:46 pm 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
cURL Sample 2:

Is it true that the following short code is just as good as the one mentioned on my previous post ?

Code:

<?php

//This code was found on: http://www.binarytides.com/php-curl-tutorial-beginners/
//2nd Example
//The above GET request to a url can be done in a much simpler way like this:

//Make a HTTP GET request and print it (requires allow_url_fopen to be enabled)
echo file_get_contents('http://www.apple.com/');

?>



Last edited by UniqueIdeaMan on Tue May 23, 2017 5:56 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Tue May 23, 2017 5:50 pm 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
cURL Sample 3:

Can you figure-out or atleast guess why the following code is better than the previous 2 ?
What benefits do you see in it than the other 2 code samples ?
No, I'm not testing you but trying to learn from you.

Code:

<?php

//3rd Option
//This code was found on: http://www.binarytides.com/php-curl-tutorial-beginners/
//Calling the curl_setopt function again and again to set the options is a bit tedious. There is a useful function called curl_setopt_array that takes an array of options and sets them all at once. Here is a quick example:
//Make a HTTP GET request and print it (requires allow_url_fopen to be enabled)
echo file_get_contents('http://www.apple.com/');
curl_setopt_array($ch, array(
    CURLOPT_URL => $url ,
    CURLOPT_RETURNTRANSFER => 1,
    CURLOPT_CONNECTTIMEOUT => $timeout ,
));

?>



And, what is meant by the following:
"Calling the curl_setopt function again and again to set the options is a bit tedious. There is a useful function called curl_setopt_array that takes an array of options and sets them all at once.".

Can you elaborate it ? Because, if I understand it then I'll understand the benefits of this code over the previous 2.

Thanks!


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Wed May 24, 2017 9:19 am 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
Guys,

On this cURL tutorial:
http://www.binarytides.com/php-curl-tutorial-beginners/

Under the section **"Make GET requests - fetch a url"**, you will see 3 blocks of code.
I'm referring to the 3rd one that looks like this:


Code:

    curl_setopt_array($ch, array(
        CURLOPT_URL => $url ,
        CURLOPT_RETURNTRANSFER => 1,
        CURLOPT_CONNECTTIMEOUT => $timeout ,
    ));





How come no url is mentioned in that code ?
I can now see why my code sample 3 is not working. See my 4th post above.
What do you think that code should look like ? Care to show an example, how you'd do things ?


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Sat May 27, 2017 9:30 am 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
Guys,

What I am trying to do is load google.com with cURL and no matter what link is on the page, be it a google link or another domain), I want the script to precede: http://mydomain.com/tracker.php?

So, if the page contains a link like this:

http://somedomain.com/

then that should be replaced with:

http://mydomain.com/tracker.php?http://somedomain.com/

And, if the page contains link like this:

http://subdomain.somedomain.com/dir/sub-dir/page.html

Then, I want it to be replaced to:

http://mydomain.com/tracker.php?http:// ... /page.html

You get my point ?

Here's my code but I get error:

Warning: preg_replace(): Delimiter must not be alphanumeric or backslash in C:\xampp\htdocs\test\CURL_experiment1.php on line 13

Here's my code:

Code:
<?php
$url = "http://google.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$url");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);

$pattern = 'http://';
$replacement = 'http://mydomain.com/tracker.php?';
echo preg_replace($pattern, $replacement, $result);

?>


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Tue May 30, 2017 10:01 am 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
What does the following header mean (ebay.com):

**HTTP/1.1 200 OK X-EBAY-C-REQUEST-ID: ri=N79YSYGvO9se,rci=qZJNjVUUeUF9xCaS RlogId: t6e%60cckjkb9%3Fvo%7Bccbgmijf%28vo%7B%287570577-15c59d3449c-0x129 X-Frame-Options: SAMEORIGIN X-Frame-Options: SAMEORIGIN X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Cache-Control: private Pragma: no-cache Content-Type: text/html;charset=utf-8 Content-Language: en-US Server: ebay server X-EdgeConnect-MidMile-RTT: 172 X-EdgeConnect-Origin-MEX-Latency: 261 X-EdgeConnect-Cache-Status: 0 Date: Tue, 30 May 2017 14:47:17 GMT Transfer-Encoding: chunked Connection: keep-alive Connection: Transfer-Encoding Set-Cookie: JSESSIONID=B9699C4784AF48E8C158816A1094F78B; Path=/; HttpOnly Set-Cookie: ebay=%5Esbf%3D%23%5E;Domain=.ebay.com;Path=/ Set-Cookie: dp1=bu1p/QEBfX0BAX19AQA**5b0eb975^bl/BD5cefecf5^;Domain=.ebay.com;Expires=Thu, 30-May-2019 14:47:17 GMT;Path=/ Set-Cookie: s=CgAD4ACBZLtd1NTlkMzQ0OGMxNWMwYTg4OGQ4YzYzYTYzZmZmOGJhNjhjlJyS;Domain=.ebay.com;Path=/; HttpOnly Set-Cookie: nonsession=CgADLAAFZLYz9MQDKACBik4d1NTlkMzQ0OGMxNWMwYTg4OGQ4YzYzYTYzZmZmOGJhNjgpg9Wr;Domain=.ebay.com;Expires=Wed, 30-May-2018 14:47:17 GMT;Path=/**


I have a feeling the header states that the visitor is the same visitor and so is this the case why I'm unable to precede the following on all links present on ebay homepage ?
(NOTE: I'm getting cURL to fetch ebay homepage).

http://mydomain.com/tracker.php?

Check the following code. Run it on your xamp/wamp and then hover your mouse over the links present on ebay's homepage. Do you see all links starting with:

http://mydomain.com/tracker.php?

Yes, or no ? i used to see it. And so a YES from here from lastnight.
But tonight, the answer is a NO.
Now, I'm confused why is that ? Anything wrong with my coding ?

Code:
<?php
$url = "http://www.ebay.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$url");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);

/* pattern can be either 2:
$pattern = '#http://#';
$pattern = '/http:\/\//';
*/

$pattern = '/http:\/\//';
$replacement = 'http://mydomain.com/tracker.php?';

$pattern = '/https:\/\//';
$replacement = 'http://mydomain.com/tracker.php?';

$pattern = '/localhost\//';
$replacement = 'http://mydomain.com/tracker.php?';

echo str_replace($pattern, $replacement, $result);

?>



What I'm doing here is just replacing the "http://" with:

http://mydomain.com/tracker.php?

But instead of using the replace function, is there a better way to precede the following on all links found on ebay homepage ? (The technique should work on any homepage and not be dependant on ebay homepage only).


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Sat Jun 17, 2017 10:13 am 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
Folks,

I'm afraid the following preg_replace is not doing the job. Can you think of a better one ?
Actually, if you can come-up with one that does the following then I'd appreciate it:

1. Replace 'https://' with: 'http://mydomain.com'.
2. Replace 'http://' with: 'http://mydomain.com'.
3. Replace 'www.' with: 'http://mydomain.com'.
4. Replace all subdomains and sub sub domains etc. with: 'http://mydomain.com'.
eg1. Replace 'mail.domain.com' with 'http://mydomain.com'.
eg2. Replace 'ny.mail.domain.com' with 'http://mydomain.com'.
eg3. Replace 'europe.spain.mail.domain.com' with 'http://mydomain.com'.
eg4. Replace 'west.europe.deutchland.mail.domain.com' with 'http://mydomain.com'.
And so on. You get the picture.

I tried the following code but it's not working. Instead of replacing things with 'http://mydomain.com' it is replacing with: '%24url//'. And that is a big No! No!

Code:

<?php
$url = "http://google.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "$url");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_HEADER, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
$result = preg_replace("#(<\s*a\s+[^>]*href\s*=\s*[\"'])(?!http)([^\"'>]+)([\"'>]+)#",'$1http://$url/$2$3', $result);
echo $result
?>



Cheers!


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Sun Jun 18, 2017 11:59 am 
Offline
Forum Contributor
User avatar

Joined: Sat Nov 20, 2010 10:00 pm
Posts: 241
Location: GA, USA
preg_replace does just fine. Though I tend to use preg_replace_callback instead, so I can use an anonymous function to handle the replacements

_________________
- Kaleb Klein
------------------------------------
Web Developer | Software Developer
https://kalebklein.com
PGP Key: https://keybase.io/pazuzu156


Top
 Profile  
 
 Post subject: Re: cURL Experiments
PostPosted: Mon Sep 18, 2017 4:59 pm 
Offline
Forum Contributor

Joined: Wed Jan 18, 2017 4:43 pm
Posts: 163
Pazuzu156 wrote:
preg_replace does just fine. Though I tend to use preg_replace_callback instead, so I can use an anonymous function to handle the replacements


How about showing an example ? Take my code, for starters, and build your code sample from there.

Thanks!


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 17 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group