help with url query from PHP with cURL

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
elann
Forum Newbie
Posts: 2
Joined: Tue Aug 03, 2010 12:03 pm

help with url query from PHP with cURL

Post by elann »

Hi all,

I'm new to the PHP game and I'm working on a one off project. I'm trying to write a php file that will (1) submit a url query to a website, (2) open the downloaded csv file and (3) save it to a my sql database. So far I've gotten stages 2 and 3 to work. I'm not sure how to get stage 1 to work.

the following is the url which causes a csv file to be downloaded:

http://www.eirgrid.com/operations/syste ... eneration/

I've no idea where to go with this. One approach I was messing with was a curl implementation along the lines of:

Code: Select all

$url='http://www.eirgrid.com/operations/systemperformancedata/download.jsp';
		$fields=array(
					
					'download'=>'Y',
					'startdate'=>'31/07/2010',
					'enddate'=>'31/07/2010',
					'proc'=>'data_pack.getwindgenforadayt4',
					'templatename'=>'Wind%20Generation',
					'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW'	,
					'prevurl'=>'http://www.eirgrid.com/operations/systemperformancedata/windgeneration/'
					);
		
		
		
		//url-ify the data for the POST 
		foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; } 
		echo $fields_string.'<p></p>';
		
		$fields_string=rtrim($fields_string,'&');
		
		echo $fields_string.'<p></p>';
		$ch = curl_init();
		
		echo $ch;

		//set the url, number of POST vars, POST data 
		curl_setopt($ch,CURLOPT_URL,$url);
		curl_setopt($ch,CURLOPT_POST,count($fields));
		curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
		
		// grab URL and pass it to the browser
		$result=curl_exec($ch);
		var_dump($result);
		
		// close cURL resource, and free up system resources
		curl_close($ch);
This isn't yielding the expected result. I'm not fully au fait with either PHP or cURL so this may be totally unsuited. Maybe some of you might be able to set me in the right direction with this to get the file to download.

Thanks!
bibumathew
Forum Newbie
Posts: 8
Joined: Tue Aug 03, 2010 5:36 pm

Re: help with url query from PHP with cURL

Post by bibumathew »

Hope this will help you?
if you have any questions let me know


$url='http://www.eirgrid.com/operations/syste ... wnload.jsp';
$fields=array(

'download'=>'Y',
'startdate'=>'31/07/2010',
'enddate'=>'31/07/2010',
'proc'=>'data_pack.getwindgenforadayt4',
'templatename'=>'Wind%20Generation',
'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW' ,
'prevurl'=>'http://www.eirgrid.com/operations/syste ... eneration/'
);



//url-ify the data for the POST
$fields_string=NULL;
foreach($fields as $key=>$value) {
$fields_string .= $key.'='.$value.'&';
}


$fields_string=rtrim($fields_string,'&');

$newfilename="download.tsv";
$out = fopen($newfilename, 'wb');
$ch = curl_init();




//set the url, number of POST vars, POST data
curl_setopt($ch, CURLOPT_FILE, $out);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);

// grab URL and pass it to the browser
$result=curl_exec($ch);





// close cURL resource, and free up system resources
fclose($out);
curl_close($ch);

if u want to download the csv file and store it in a location then u have to set it in the curl_setopt

when u run this program ur out put file will be saved in download.tsv
bibumathew
Forum Newbie
Posts: 8
Joined: Tue Aug 03, 2010 5:36 pm

Re: help with url query from PHP with cURL

Post by bibumathew »

Hope this will help you?
if you have any questions let me know

Code: Select all

$url='http://www.eirgrid.com/operations/systemperformancedata/download.jsp';
                $fields=array(
                                       
                                        'download'=>'Y',
                                        'startdate'=>'31/07/2010',
                                        'enddate'=>'31/07/2010',
                                        'proc'=>'data_pack.getwindgenforadayt4',
                                        'templatename'=>'Wind%20Generation',
                                        'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW'      ,
                                        'prevurl'=>'http://www.eirgrid.com/operations/systemperformancedata/windgeneration/'
                                        );
               
               
               
                //url-ify the data for the POST
                $fields_string=NULL;
                foreach($fields as $key=>$value) { 
                	$fields_string .= $key.'='.$value.'&'; 
                }
         
               
                $fields_string=rtrim($fields_string,'&');

                $newfilename="download.tsv";
                $out = fopen($newfilename, 'wb');                    
                $ch = curl_init();
                 
               
              

                //set the url, number of POST vars, POST data
                curl_setopt($ch, CURLOPT_FILE, $out); 
                curl_setopt($ch,CURLOPT_URL,$url);
                curl_setopt($ch,CURLOPT_POST,count($fields));
                curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
             
                // grab URL and pass it to the browser
                $result=curl_exec($ch);
                  
              
                
            
               
                // close cURL resource, and free up system resources
                fclose($out);
                curl_close($ch);
if u want to download the csv file and store it in a location then u have to set it in the curl_setopt

when u run this program ur out put file will be saved in download.tsv

Code: Select all

[syntax=php]
[/syntax]
elann
Forum Newbie
Posts: 2
Joined: Tue Aug 03, 2010 12:03 pm

Re: help with url query from PHP with cURL

Post by elann »

Thank you, thank you, thank you, thank you.

I'm embarrassed how close I was.
Post Reply