Optimizing Dynamic Pages - Part II

Optimizing Dynamic Pages - Part II

The Widget Queen Revisited
You have the world's finest collection of widgets. You created the world's best
widget website. You have no traffic.

You checked in the search engines and find that your site does not appear at
all, even though all your competitors' sites do. Perhaps the search engine
robots cannot get to your pages to index them.

Search Engine Robots
Search engine robots are simple creatures. They can "read" text to add to their
databases, and they can follow "normal" links--those links that are coded to
look like

blue widgets

or the slight variation



That's it. Search engine robots cannot select items from lists; search engine
robots cannot type text into boxes; search engine robots cannot click "submit"
buttons. That means that no matter how important our dynamically-generated page
of blue widgets is, if the only way to access that page is to select it from a
list or click on a button, the robot will never be able to visit it. That, in
turn, means that it will never appear in the search engine results.

So how do you get your dynamic information to show up in non-dynamic ways?

The Painful Solution
One of the reasons that dynamic pages exist is because of the difficulty
involved in constantly updating -- adding and deleting -- pages from your site,
based on which widgets you are offering this season. If you have a separate page
for each make and model of widget, each of those pages can be spidered. They can
all be reached through links that look like

blue widgets style 1
blue widgets style 2
red widgets style 1
red widgets style 2
new widgets style 1
new widgets style 2

The bad news here, of course, is that you now have to create all of those pages.
This loses the benefit of drawing the widget information from a database.

A Better Solution
A better solution is to create only a "shell" of each page, and then to
dynamically populate the page from our database. By creating a "real" file, you
can assign a fixed URL, but still use the database to fill-in the page, using
any of various server-side techniques (HTML server-side includes, Perl, Active
Server Pages, Java Server Pages, PHP, etc.). A simple page like this might
suffice: