Page 1 of 2
extract information from url-txt
Posted: Tue Nov 19, 2002 2:00 pm
by soshea
being new to php i was wondering about the best tactic to extract information from txt file on the internet.
the jist of it is this, i need to get just the line above "end of file" that has the latest information on snowdepth:
02-11-19 08:00 1.7 13.6
02-11-19 09:00 2.0 13.5
02-11-19 10:00 3.4 13.1
02-11-19 11:00 5.8 11.8
end of file
which is updated every hour. and i need to make the date , temperature(second variable), snowdepth (third variable) each a string. i.e convert the temp from Cel to Fer ect.....
i have looked at eregi and preg_match but was wondering what the experts would utilize and the route they would take.
the actual txt is this:
ftp://ftp.wcc.nrcs.usda.gov/states/id/d ... us_ski.txt
thanx for any help!
sean
Posted: Tue Nov 19, 2002 2:17 pm
by BDKR
Cool stuff. Makes me miss snow boarding back in Tahoe. Don't do much of that here in the Caribean.
Anyways, I looked at the file and it helps that it actually says "end of file" there at the bottom. Also, it's good that the file isn't that big. What I would suggest is using file(). Check out what I did below.
Code: Select all
<?php
$file_array=file("ftp://ftp.wcc.nrcs.usda.gov/states/id/depth/bogus_ski.txt");
print_r($file_array);
?>
file() reads the contents of a file and stores each line as an element in an array. Now all you have to do is process the array. Seek to the element containing "end of file" then back up one.
Check out the manual and post your code here if you need some more help.
Cheers,
BDKR
Posted: Tue Nov 19, 2002 2:29 pm
by soshea
ahhh, tahoe. i actually spent more time at royal gorge if in cal or nev.. but bachelor and hood were always home base. not much snow in the caribean, but i am sure the scenery is nice..
thanx for the advice, free time for this isn't until tonight, just wanted to jump start it.
sean
Posted: Tue Nov 19, 2002 2:43 pm
by BDKR
ahhh, tahoe.
Yeah..... Those were good times. The Subaru, powerbars, and the lights reflecting off the water.
Well, I ahd better get back to work and stop daydreaming.
Drop a note if you need some more help.
Cheers,
BDKR[/quote]
Posted: Tue Nov 19, 2002 11:16 pm
by soshea
got the in_array(), but could not findout how to go back 1. also when you do find the value how do you find out its key. in the examples i have seen some will utilize $k for key and others utilize $key (same with the values)
Code: Select all
<?php
//got the quicker page than ftp
$file_array = file("http://idsnow.id.nrcs.usda.gov/snow/pub/bogus_ski.txt");
//trim the values in the array
$snowdepth = array_map("trim", $file_array);
//search for end of file
if(in_array("end of file", $snowdepth))
{
//the code to go back one key here
print ("it was found")
}
?>
thanx for the help!!
Posted: Wed Nov 20, 2002 2:33 am
by twigletmac
If you use
array_search() it will return the key if the value is found then you can just subtract 1 from that value to get the key for the value you want.
Mac
Posted: Wed Nov 20, 2002 5:51 am
by cybaf
I would read the file into an array and then use:
Code: Select all
<?php
$arrayCount = count($file_array);
echo $file_arrayї$arrayCount-1];
?>
this would echo the second last row of the file
//Anders
Posted: Wed Nov 20, 2002 6:25 am
by BDKR
cybaf wrote:
I would read the file into an array and then use:
Code: Select all
<?php
$arrayCount = count($file_array);
echo $file_arrayї$arrayCount-1];
?>
this would echo the second last row of the file
//Anders
True, but the only problem wth this is that the line "end of file" isn't actually the end of the file. If you look at the URL posted, there are another 3 or 4 lines below it. Therefore, it's essential to locate that "end of file' line as a reference.
Cheers,
BDKR
Posted: Wed Nov 20, 2002 11:32 am
by soshea
alright, you all are making my brain hurt for a quad mocha. here is what i am understanding with the books and the helpful suggestions:
Code: Select all
<?php
<?php
//got the quicker page than ftp
$file_array = file("http://idsnow.id.nrcs.usda.gov/snow/pub/bogus_ski.txt");
//trim the values in the array
$snowdepth = array_map("trim", $file_array);
//search for end of file
$searchsnow = array_search("end of file", $snowdepth);
echo $file_arrayї$searchsnow-1];
?>
just tested and it worked. now that i have the string of the latest information, i am going to take the string created another array so that i can single out the temperature to convert from C to F, and take the depth and convert from inches to feet for another program. i was going to utilize the explode() function.
thanx all for the help, out of the other forums for other languages, this is one of the better ones!!!!

Posted: Fri Nov 22, 2002 8:07 am
by BDKR
...just tested and it worked
Good show!
Posted: Fri Nov 22, 2002 1:25 pm
by soshea
thanx for the suggestions and help: if you have any tips-advice for the next challenge by all means forward them!!
the next chanllenge is this:
ftp://ftp.wcc.nrcs.usda.gov/states/id/depth/depth1.txt
this file has the data for the rest of the areas, what i am stuck on is removing the blank lines after the last data of information. with all of the snow markers on this single page i am going to attempt to grab the marker name and the most recent entry of data ie:
BOGUS BASIN 11/22/02 09:00 4.1 2.3 6.7 15.4
11/22/02 10:00 4.1 2.3 10.1 15.4
and make it:
BOGUS BASIN 11/22/02 10:00 4.1 2.3 10.1 15.4
for all depth markers.
my concept is to set a cron job to run every 30 min with the php file that will get the .txt, parse out the information that i need, and store it as a file on the local webserver. that way it will make the users pages quicker.
here has been my latest attempt to remove \n\r and format the data (made the data even worse):
Code: Select all
<?php
// first attempt to get rid of newline and car
$file_array = file("depth1.txt");
$stringsnow = implode("",$file_array);
$arraysnowdepth = trim($stringsnow, "\n\r");
print "$arraysnowdepth";
$snowdepth = explode(" ", $arraysnowdepth);
while (list($k, $v) = each ($snowdepth))
{
print("<br>key: " .$k. "value: " .$v);
}
?>
thanx for any advice!
Posted: Fri Nov 22, 2002 2:59 pm
by BDKR
OK Sosh...,
Most of your problem is really a logic issue. The cool thing here is that you're giving it a run go as opposed to saying "Write this for me".
Anyways, the first three lines of code are kinda strange in an eliptical kinda way. You first create an array of the file, the you implode it back to a file, then you try to trim the end line chars, which is something that should be done line by line.
I'm not sure how to explain it, but it's a little strange.
Going on, I (re?) wrote some, or maybe just enough to try to help you see a way out of the woods here.
Code: Select all
<?php
$file_array = file("depth1.txt");
# Get rid of newline characters
while(list($key, $val)=each($file_array))
{
$val=trim($val, "\n");
$val=trim($val, "\r");
}
# Move back to element 0
reset($file_array);
# Create references for our search
$reference="******************** WATER PREC Deg.C DEPTH ";
$end="-----------------------Temperature Conversions-------------------------------";
# Now parse according to what we want to see
while(list($key, $val)=each($file_array))
{
if(strstr($val, $reference))
{ echo $file_arrayї$key-1]; echo "\n"; sleep(1); }
if(strstr($val, $end))
{ break; }
}
?>
First off, the best thing you can do when needing to parse a file that's not something like XML is look at it and search for some consistency or pattern. Once you now what those are, you can search for them and make decistions based on what's found. When you run the above code, it iterates over the array and when it finds that $reference point, it echos the name of a marker (?) which is a just one line above it. Also, if it comes across the $end marker, it knows there's nothing else below that point of any use and breaks out of the loop.
Now that should get you going in a good direction. It's not all of it, but it's easily enough to help you get the rest sorted.
One last thing. If you are going to run this in a browser, I would suggest getting rid of the sleep() function. That was just a way to show how the script is operation.
Cheers,
BDKR
changing direction
Posted: Mon Nov 25, 2002 11:39 am
by soshea
sorry about the delay on the reply, i got my eyes cut on and have rid myself of glasses.
while taking a break i realized that i was going about this the wrong way. instead of getting all the snow markers, i am going to get only the ones that really matter to what is needed (closest to the ski areas). i did get close on getting the all areas workiing, the problem i ran into is that there is an extra carriage return on the first area after the snow report so all others had 1 blank line separating them, but this one had 2. so everything was a bit offset, results would show on the 3rd snowmarker, but that was actually data for the second snow marker and the 1 would not have any data(not to make this confusing).
Dynamically making this happen would be great (especially if a new marker is added), but as of current, this .txt will stay the same.
here is the old code:
Code: Select all
<?php
$file_array = file("depth1.txt");
# $file_array = trim($file_array);
# Get rid of newline characters
while(list($key, $val)=each($file_array))
{
$val=trim($val, "\n");
$val=trim($val, "\r");
}
# Move back to element 0
reset($file_array);
# Create references for our search
$reference="******************** WATER PREC Deg.C DEPTH ";
$SpacerLine=" ";
$end="-----------------------Temperature Conversions-------------------------------";
# Now parse according to what we want to see
while(list($key, $val)=each($file_array))
{
if(strstr($val, $reference))
{
$SnowMarker = str_replace('Date, PST Time SNOW AIR TEMP SNOW','',$file_arrayї$key-1]);
echo $SnowMarker;
if(strstr($SnowMarker, $SpacerLine))
{
echo $file_arrayї$key-3];
}
echo "\n";
print "<br>";
} #sleep(1);
if(strstr($val, $end))
{
break;
}
}
?>
if you would like i will post the new code so that you can suggest changes to my code. i have already learned alot from your prior tips!
Posted: Mon Dec 02, 2002 7:26 am
by BDKR
Sorry I haven't gotten back to you. I've been a little busy and hot about certain things going on elsewhere on the net.
Anyways, what's your status on this project? I've lost the plot a little and need to be updated.
Cheers,
BDKR
Posted: Tue Dec 03, 2002 12:05 am
by soshea
currently i am working on copying the $file_array that we got off of the .txt file and writing it to a local file on my own box. that way when people come to the site the current conditions page will not take a while to load. i am going to setup a cron job that will execute a .php file which will delete and rewrite the the .txt file every 15 min. and then have the web code parse the .txt on the local box when the people visit.
the ouch has been the "Warning: fopen("bogus_ski.txt", "w") - Permission denied in /var/www/html/tests/copyfile.php on line 4". i looked through the php.ini and am reading about what modifications need to be made.
unfortunately i spent most of the weekend getting the site formatted so i could focus on the php this week.
thanx for the followup!
soshea