Search Engine Spiders and Dynamic Websites

XML, Perl, Python, and other languages can be discussed here, even if it isn't PHP (We might forgive you).

Moderator: General Moderators

Post Reply
trent2800
Forum Commoner
Posts: 48
Joined: Mon Oct 02, 2006 7:02 am

Search Engine Spiders and Dynamic Websites

Post by trent2800 »

It was recently brought to my attention that search engine spiders will 'choke' on dynamically generated pages. However, I couldnt find a more recent article than about 2003 so, I'm wondering if this is still valid, and if so, does anyone have any resources on how to use filesystems instead of query strings.

Thanks,

Tyler
User avatar
feyd
Neighborhood Spidermoddy
Posts: 31559
Joined: Mon Mar 29, 2004 3:24 pm
Location: Bothell, Washington, USA

Post by feyd »

They haven't had "problems" with dynamic pages in quite some time.
User avatar
John Cartwright
Site Admin
Posts: 11470
Joined: Tue Dec 23, 2003 2:10 am
Location: Toronto
Contact:

Post by John Cartwright »

On dynamic pages, such as wordpress blogs which feature n amount of articles on the main page, they will attach a permalink (permanent link) in which links to that particular content reguardless if that content has moved elsewhere.
User avatar
pedrotuga
Forum Contributor
Posts: 249
Joined: Tue Dec 13, 2005 11:08 pm

Post by pedrotuga »

There is not that many "dynamic pages".
Pages are dynamic, but they usualy are associated with an url that in most of the cases will return the same page is called after some time.
Anyway i still find a lot of brocken references... i use google's cache in like.... i dunno... about 20% of my web browsing.
Post Reply