Duplicate content is a big problem for both webmasters and search engines. It stems from duplicate versions of the same content being accessible via different URLs.

Duplicate content is widely used in Black Hat SEO to deceive search engines into indexing extra non-original content in an attempt to gain more exposure. But search engines may blacklist your site for non-intentional use of duplicate content.

An example of duplicate content issues can be seen as follows:

www.example.com
example.com
www.example.com/index.asp
www.example.com/INDEX.asp
example.com/index.asp?ref=10

All of the above URLs would load the same content, which makes it hard for a search engine to determine which is your 'master' URL and also means that users may be linking to your content via multiple URLs.

There are different ways to fix the issue, including updating your CMS or creating an XML sitemap of all your website URLs, but the three we're going to cover here are:

301 redirects: This method permanently redirects a particular URL to another. It's typically used when a website has moved server or the URL structure has changed. Configuring 301 redirects depends on the website/web server set-up. Check out this guide to the most common environments.

Canonical tag: In February, Google, Yahoo and Microsoft announced support for a new HTML link element to combat duplicate content. The canonical link element tells search engines which version of the content is to be considered as the master.

Canonical tags

To implement the canonical element, you need to place the following code in the head section of your page template: <link rel="canonical" href="http:// example.com/page.html"/>. This tells search engines only to index the master URL.

Pre-built plug-ins: A number of pre-build plugins are available for popular CMSs, which assist in remedying any duplicate content issues, including WordPress, Drupal and Magento.

The future of SEO

The search engine landscape continues to become more complex and intriguing. The launch of new search engines such as Wolfram Alpha and Bing; the partnership agreement between Microsoft and Yahoo; Google's 'Caffeine' update; Facebook's purchase of FriendFeed …

But what does all this mean for SEO? Do we need to develop advanced techniques to keep up with the advances in search technology? In a nutshell … no.

Stick to the basics, create useful content, build on an accessible and adaptable platform, follow common sense and best practice when creating content and you'll fare well.

Essentially, search engines are trying to behave naturally. If your site is more useful than your competitors' and answers your users' queries better, then it's in the search engine's best interest to promote your site.

-------------------------------------------------------------------------------------------------------

First published in .net Issue 195

Liked this? Then check out How to create your best website layouts ever

Sign up for TechRadar's free Weird Week in Tech newsletter
Get the oddest tech stories of the week, plus the most popular news and reviews delivered straight to your inbox. Sign up at http://www.techradar.com/register

Follow TechRadar on Twitter