Mar
17
With online content becoming more dynamic, are search engines able to categorize and serve up dynamic content?
March 17th, 2010 posted by
admin
Are search engines like google and yahoo able to categorize a page that doesn’t exist until variables are introduced? How does a spider work within a dynamic site?
IT Pro says:
March 20th, 2010 at 4:50 pm
They are able to do so.
Some search engines may shy away from sites that have variables in their URLs, such as
You can use mod_rewrite in Apache to trick engines into crawling sites that have variables in thier URLs. For example, the above line would be rewritten as.
BountyJack says:
March 23rd, 2010 at 12:39 am
Dynamic content is delivered to the Web browser in a different form than it exists on the server, while static content is stored on the Web server in the same format that is delivered to the Web browser. Dynamic site pages are generated from a database “on the fly” as users request them.
You can often tell when you are looking at a dynamically generated page, because dynamic URLs contain one or more “query strings,” or question marks (?) while static URLs do not, but there are exceptions to this rule, which we shall discuss below.
Search engines have a hard time with dynamic URLs. Dynamic URLs may cause search engines to mistake a small site for a very large one because an unlimited number of URLs can be used to provide essentially the same content. This can cause search engine spiders to avoid such sites for fear of falling into “dynamic spider traps,” crawling through thousands of URLs when only a few are needed to represent the available content.