Page 1 of 1

indexing a webbsite

Posted: Wed Oct 19, 2005 6:54 am
by pelegk1
i am lookign for a free code that index the pages of a website
so when i searc for a keword on my websiite i get results (like google for example does(
does any 1 know of a good and free ready made scruipta?
thnaks in advance
peleg

Posted: Wed Oct 19, 2005 6:58 am
by feyd
hotscripts.com likely has some...

Posted: Wed Oct 19, 2005 10:01 am
by pickle
Is your website content stored in a database or in flat files?

Re: indexing a webbsite

Posted: Wed Oct 19, 2005 11:17 am
by foobar
pelegk1 wrote:i am lookign for a free code that index the pages of a website
so when i searc for a keword on my websiite i get results (like google for example does(
does any 1 know of a good and free ready made scruipta?
thnaks in advance
peleg
Not sure off the top of my head if Google search on your own website is free, but I'd check it out. If the customizable package isn't, what you can do is make your search form submit to Google.

If you don't want to use a proprietary app/script, which shows good initiative, try figuring out something for your own website. Like a CronJob that indexes all new static pages every once in a while and saves the results in a relational database for speed. I would recommend a Regex that filters out the keywords & description of a page, much like what Google does. Optionally, you could of course search through the entire page for certain keywords, such as unique words, author info, etc. (check Wikipedia for the system Amazon.com uses, forgot the name).

yes i have a db

Posted: Thu Oct 20, 2005 4:54 pm
by pelegk2
pickle wrote:Is your website content stored in a database or in flat files?
that holds the content ofthe pages

Re: yes i have a db

Posted: Thu Oct 20, 2005 5:02 pm
by pickle
pelegk2 wrote:
pickle wrote:Is your website content stored in a database or in flat files?
that holds the content ofthe pages
Um.. ok. That doesn't really answer my question though. Indexing your site will take two drastically different routes depending on whether your website content is stored in individual files (such as individual .php files or .html files) or if its stored in a database.

well pickle

Posted: Fri Oct 21, 2005 7:59 am
by pelegk2
some of it is in a databse but most of it in php files

Posted: Fri Oct 21, 2005 11:03 am
by pickle
Well, the database stuff can be easily searched by setting up a FULLTEXT index and searching with that.

Looking through the files might be more difficult. Try Googling or looking at Hotscripts. You're going to want a script that opens the directory, looks for each file, then indexes each of those files. The problem you're going to have is making the indexing script separate content from PHP code.

Posted: Fri Oct 21, 2005 3:41 pm
by chrys
If you mean indexing it to be searched... check out htdig