Ajax software     Free javascripts 
↑


Main Page

Some Thoughts

Which method do we suggest in most cases? The first method — exclusion. If you look closely at the fol-

lowing two links on a web site created by one of the authors of this book:

http://www.lawyerseek.com/Practice/In-the-News-C20/Protopic-P38/

http://www.lawyerseek.com/Practice/Pharmaceutical-Injury-C1/Protopic-P38/

There are two URLs, but one is excluded in the

robots.txt

file. The former is excluded. The

robots.txt

file at

http://www.lawyerseek.com/

contains the following entry that excludes the following URL:

User-agent: *

...

...

Disallow: /Practice/In-The-News-C20/Protopic-P38/

Similar Pages

If you have several very similar products that exist on multiple URLs, think about changing your web

application to contain the various permutations of the products on one page. Consider the example of a

product that comes in several different colors. The resulting product pages would typically contain differ-

ent pictures but substantially duplicate descriptions. The business logic may dictate that these are different

products with different SKUs, but you can still present them on one page with a pull-down menu to select

the color/SKU to be added to the shopping cart.

The shopping cart page of an e-commerce site, login pages, and other like pages should also not be

indexed, because there is typically no valuable content on such pages. For example, it is easy to see

how the following shopping cart URLs could create duplicate content:

http://www.example.com/cart.php?product_id=1

...

...

http://www.example.com/cart.php?product_id=99

Pages with Duplicate Meta Tag or Title Values

A common mistake is to set the

meta keywords

,

meta description

, or title values on a web site to the

same default value programmatically for every page. Aaron Wall of SEOBook states

“If you have complete

duplication of any element (page title, meta keywords, meta description) across your site then it is at best a wasted

opportunity, but may also hurt your ability to get your site indexed or ranked well in some search engines.”

If

time and resources cannot be dedicated to creating unique meta tags, they should probably not be

created at all, because using the same value for every page is certainly not beneficial. Having identical

titles for every page on a web site is also particularly detrimental. Many programmers make this mis-

take, because it is very easy not to notice it.

URL Canonicalization

Many web sites exhibit subtle but sometimes insidious duplicate problems due to URL canonicalization

problems. The two most common of such problems are documented in Chapter 4 in the sections “URL

Canonicalization: www.example.com versus example.com,” and “URL Canonicalization: /index.php

versus /.”

106

Chapter 5: Duplicate Content

c05.qxd:c05 10:41 106

Ajax software     Free javascripts 
→