general directory crawling problems

PHP programming forum. Ask questions or help people concerning PHP code. Don't understand a function? Need help implementing a class? Don't understand a class? Here is where to ask. Remember to do your homework!

Moderator: General Moderators

Post Reply
drumking88
Forum Newbie
Posts: 9
Joined: Tue Apr 28, 2009 4:32 pm

general directory crawling problems

Post by drumking88 »

I am trying to get a searcher working as part of a project and am a little confused. I need to be able to get the html content from all the files in the directory, sub director(ies) and dump it into a mysql database. I have no problems connecting to a database, but I think this code is causing problems at the start when doing the initial database dump.

Code: Select all

 
$dir = '.';
$files = scandir($dir);
$files1 = array_slice($files, 2);
 
$testArray = array(1 =>$files);
$delete = mysql_query("DELETE FROM spider");
foreach ($files1 as $page) {
$content = file_get_contents($page);
$scontent = strip_tags($content);
 
This works fine for some files when I dump the findings into a mysql database, but in other files, it just screws up incredibly.

any help would be much appreciated.

thanks
Last edited by Benjamin on Tue Apr 28, 2009 4:38 pm, edited 1 time in total.
Reason: Added code tags, type = php.
Post Reply