Page 1 of 1

help with url query from PHP with cURL

Posted: Tue Aug 03, 2010 3:51 pm
by elann
Hi all,

I'm new to the PHP game and I'm working on a one off project. I'm trying to write a php file that will (1) submit a url query to a website, (2) open the downloaded csv file and (3) save it to a my sql database. So far I've gotten stages 2 and 3 to work. I'm not sure how to get stage 1 to work.

the following is the url which causes a csv file to be downloaded:

http://www.eirgrid.com/operations/syste ... eneration/

I've no idea where to go with this. One approach I was messing with was a curl implementation along the lines of:

Code: Select all

$url='http://www.eirgrid.com/operations/systemperformancedata/download.jsp';
		$fields=array(
					
					'download'=>'Y',
					'startdate'=>'31/07/2010',
					'enddate'=>'31/07/2010',
					'proc'=>'data_pack.getwindgenforadayt4',
					'templatename'=>'Wind%20Generation',
					'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW'	,
					'prevurl'=>'http://www.eirgrid.com/operations/systemperformancedata/windgeneration/'
					);
		
		
		
		//url-ify the data for the POST 
		foreach($fields as $key=>$value) { $fields_string .= $key.'='.$value.'&'; } 
		echo $fields_string.'<p></p>';
		
		$fields_string=rtrim($fields_string,'&');
		
		echo $fields_string.'<p></p>';
		$ch = curl_init();
		
		echo $ch;

		//set the url, number of POST vars, POST data 
		curl_setopt($ch,CURLOPT_URL,$url);
		curl_setopt($ch,CURLOPT_POST,count($fields));
		curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
		
		// grab URL and pass it to the browser
		$result=curl_exec($ch);
		var_dump($result);
		
		// close cURL resource, and free up system resources
		curl_close($ch);
This isn't yielding the expected result. I'm not fully au fait with either PHP or cURL so this may be totally unsuited. Maybe some of you might be able to set me in the right direction with this to get the file to download.

Thanks!

Re: help with url query from PHP with cURL

Posted: Tue Aug 03, 2010 6:03 pm
by bibumathew
Hope this will help you?
if you have any questions let me know


$url='http://www.eirgrid.com/operations/syste ... wnload.jsp';
$fields=array(

'download'=>'Y',
'startdate'=>'31/07/2010',
'enddate'=>'31/07/2010',
'proc'=>'data_pack.getwindgenforadayt4',
'templatename'=>'Wind%20Generation',
'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW' ,
'prevurl'=>'http://www.eirgrid.com/operations/syste ... eneration/'
);



//url-ify the data for the POST
$fields_string=NULL;
foreach($fields as $key=>$value) {
$fields_string .= $key.'='.$value.'&';
}


$fields_string=rtrim($fields_string,'&');

$newfilename="download.tsv";
$out = fopen($newfilename, 'wb');
$ch = curl_init();




//set the url, number of POST vars, POST data
curl_setopt($ch, CURLOPT_FILE, $out);
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);

// grab URL and pass it to the browser
$result=curl_exec($ch);





// close cURL resource, and free up system resources
fclose($out);
curl_close($ch);

if u want to download the csv file and store it in a location then u have to set it in the curl_setopt

when u run this program ur out put file will be saved in download.tsv

Re: help with url query from PHP with cURL

Posted: Tue Aug 03, 2010 6:03 pm
by bibumathew
Hope this will help you?
if you have any questions let me know

Code: Select all

$url='http://www.eirgrid.com/operations/systemperformancedata/download.jsp';
                $fields=array(
                                       
                                        'download'=>'Y',
                                        'startdate'=>'31/07/2010',
                                        'enddate'=>'31/07/2010',
                                        'proc'=>'data_pack.getwindgenforadayt4',
                                        'templatename'=>'Wind%20Generation',
                                        'columnnames'=>'Time,Wind%20Generation%20MW,Forecast%20MW'      ,
                                        'prevurl'=>'http://www.eirgrid.com/operations/systemperformancedata/windgeneration/'
                                        );
               
               
               
                //url-ify the data for the POST
                $fields_string=NULL;
                foreach($fields as $key=>$value) { 
                	$fields_string .= $key.'='.$value.'&'; 
                }
         
               
                $fields_string=rtrim($fields_string,'&');

                $newfilename="download.tsv";
                $out = fopen($newfilename, 'wb');                    
                $ch = curl_init();
                 
               
              

                //set the url, number of POST vars, POST data
                curl_setopt($ch, CURLOPT_FILE, $out); 
                curl_setopt($ch,CURLOPT_URL,$url);
                curl_setopt($ch,CURLOPT_POST,count($fields));
                curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
             
                // grab URL and pass it to the browser
                $result=curl_exec($ch);
                  
              
                
            
               
                // close cURL resource, and free up system resources
                fclose($out);
                curl_close($ch);
if u want to download the csv file and store it in a location then u have to set it in the curl_setopt

when u run this program ur out put file will be saved in download.tsv

Code: Select all

[syntax=php]
[/syntax]

Re: help with url query from PHP with cURL

Posted: Wed Aug 04, 2010 3:24 am
by elann
Thank you, thank you, thank you, thank you.

I'm embarrassed how close I was.