We thought it’d be useful to create a compact guide that lists some best practices that teams within Google and external webmasters alike can follow that could improve their sites’ crawlability and indexing.

There are also “pitfalls to avoid” that are listed here:

Page title tags:

• choosing a title that has no relation to the content on the page
• using default or vague titles like “Untitled” or “New Page 1″
• using a single title tag across all of your site’s pages or a large group of pages
• using extremely lengthy titles that are unhelpful to users
• stuffing unneeded keywords in your title tags

Description meta tags

• writing a description meta tag that has no relation to the content on the page
• using generic descriptions like “This is a webpage” or “Page about baseball cards”
• filling the description with only keywords
• copy and pasting the entire content of the document into the description meta tag
• using a single description meta tag across all of your site’s pages or a large group of pages

URL structure

• using lengthy URLs with unnecessary parameters and session IDs
• choosing generic page names like “page1.html”
• using excessive keywords like “baseball-cards-baseball-cards-baseballcards.htm”
• having deep nesting of subdirectories like “…/dir1/dir2/dir3/dir4/dir5/dir6/page.html”
• using directory names that have no relation to the content in them
• having pages from subdomains and the root directory (e.g. “domain.com/page.htm” and sub.domain.com/page.htm”) access the same content
• mixing www. and non-www. versions of URLs in your internal linking structure
• using odd capitalization of URLs (many users expect lower-case URLs and remember them better)

Navigation

• creating complex webs of navigation links, e.g. linking every page on your site to every other page
• going overboard with slicing and dicing your content (it takes twenty clicks to get to deep content)
• having a navigation based entirely on drop-down menus, images, or animations (many, but not all, search engines can discover such links on a site, but if a user can reach all pages on a site via normal text links, this will improve the accessibility of your site

HTML Sitemap

• letting your HTML sitemap page become out of date with broken links
• creating an HTML sitemap that simply lists pages without organizing them, for example by subject

404 page

• allowing your 404 pages to be indexed in search engines (make sure that your webserver is configured to give a 404 HTTP status code when non-existent pages are requested)
• providing only a vague message like “Not found”, “404″, or no 404 page at all
• using a design for your 404 pages that isn’t consistent with the rest of your site

Content

• writing sloppy text with many spelling and grammatical mistakes
• embedding text in images for textual content (users may want to copy and paste the text and search engines can’t read it)
• dumping large amounts of text on varying topics onto a page without paragraph, subheading, or layout separation
• rehashing (or even copying) existing content that will bring little extra value to users
• having duplicate or near-duplicate versions of your content across your site
• inserting numerous unnecessary keywords aimed at search engines but are annoying or nonsensical to users
• having blocks of text like “frequent misspellings used to reach this page” that add little value for users
• deceptively hiding text from users, but displaying it to search engines

Anchor text

• writing generic anchor text like “page”, “article”, or “click here”
• using text that is off-topic or has no relation to the content of the page linked to
• using the page’s URL as the anchor text in most cases (although there are certainly legitimate uses of this, such as promoting or referencing a new website’s address)
• writing long anchor text, such as a lengthy sentence or short paragraph of text
• using CSS or text styling that make links look just like regular text
• using excessively keyword-filled or lengthy anchor text just for search engines
• creating unnecessary links that don’t help with the user’s navigation of the site

Heading tags

• placing text in heading tags that wouldn’t be helpful in defining the structure of the page
• using heading tags where other tags like <em> and <strong> may be more appropriate
• erratically moving from one heading tag size to another
• excessively using heading tags throughout the page
• putting all of the page’s text into a heading tag
• using heading tags only for styling text and not presenting structure

Images

• using generic filenames like “image1.jpg”, “pic.gif”, “1.jpg” when possible (some sites with thousands of images might consider automating the naming of images)
• writing extremely lengthy filenames
• stuffing keywords into alt text or copying and pasting entire sentences
• writing excessively long alt text that would be considered spammy
• using only image links for your site’s navigation

Robots.txt

• allowing search result-like pages to be crawled (users dislike leaving one search result page and landing on another search result page that doesn’t add significant value for them)
• allowing a large number of auto-generated pages with the same or only slightly different content to be crawled: “Should these 100,000 near-duplicate pages really be in a search engine’s index?”
• allowing URLs created as a result of proxy services to be crawled

Social media promotion

• attempting to promote each new, small piece of content you create; go for big, interesting items
• involving your site in schemes where your content is artificially promoted to the top of these services

General promotion

• spamming link requests out to all sites related to your topic area
• purchasing links from another site with the aim of getting PageRank instead of traffic

Fairly standard stuff from the big G, nonetheless worth a look.

Domain names:

• It seems that having a domain name with a keyword may have a benefit.

File Names / File Paths / URLs:

• As with Domain name used with keywords may have a benefit.

Header tags:

• Page titles are one of the more influential things to use for SEO and should contain the keywords / phrase for that page. They should be unique for each page - do not use the same Title for multiple pages. Page titles should give a clear idea of what the page is going to be about.
• Descriptions should be unique and Informative and further expand on the page title.
• Keywords should only be relevant to the page they appear on and avoid using as many keywords in the tag as possible.

Content:

• The H1 tag should only be used once per page and should be the most important text on the page.
• H2 / H3 / H4 etc tags can be used to provide sub-headings, breaking up the content, again should be used once only on the page.

Navigation:

• Use meaningful text, preferably keywords, for the link, don’t use “click here”.
• Link title attribute should be used to describe and inform in a short sentence what will be found at the  destination page.

Images:

• Alt attributes should convey the same message as the image.  If a product, for example, describe it as well as including the product title.

Links out:

• Linking out may help get some recognition as an authority with Google based on the sites you link to.
• Only link to relevant/topical/quality/useful sites.

Content:

• Content is very important!  IF you don’t have something that can be read by visitors and bots, then you are basically wasting your time.
• You should have unique, informative and well written content.

Do not:

• keyword stuff any of the above.
• copy other sites.
• use junky or spammy content.
• fill pages up with just images.
• load content using JavaScript.
• load pages in Frames.
• have Flash only based content.

Background to this post:

The list above has come about through observation, testing and experimentation of what currently works best for the main search engines.  Results from forums and other blogs are also included once they have been tested.

If you have any observations you would like to make please comment.

Upgraded to Wordpress version 2.6 today and unfortunately, the blog collapsed giving 404, page not found errors on every post.

The site was hosted on a wind-oze server where .htaccess was not allowed and so I used a custom permalink as in /index.php/%postname%/ as the recommended work around. This is causing the problems with Wordpress 2.6 throwing errors since the /index.php/ was “forgotten about (?).

Since I have now moved to Linux hosting and can use htaccess again I decided to abandon the workarounds and get back to htaccess and redirect from my main Wordpress directory directly to the posts and remove the /index.php/ workaround.

How to fix it.

Wordpress say that if you have blank category base and tag base in your permalinks then setting them to anything will temporarily fix the problem until the next release. Then presumably you have to change them back.

How I fixed it

I decided not to use anymore workarounds and so used the following to get rid of the cause and the problem itself.

First I changed the custom permalink from index.php/%postname%/ to /%postname%/ and saved it. On the permalinks page - near the bottom Wordpress then gives you the redirect code for your htaccess file. I copied that and pasted it into the .htaccess file and, guess what, it worked. All my pages are now bearing the new urls and if I use a search engine for one of my blog posts and click it, that gets redirected too, and all without the /index.php/ extra!

In the meantime Wordpress are working on release 2.6.1 to fix the problem.

You have to register to use it, but it is free to use. You get a 21 day free trial of the full version, but there are several versions available, so it’s well worth a look.

I have several sites with stats and took hits from each to show the percentages of usage across them all. The sites used show a cross section of industries, interests and popularity and represent data from 200,000 hits plus. I used only one months data, that being May 2008.

So, without more ado, the stats:

Those nice people at Mozilla, as well as releasing version 3 of the Firefox browser in the next few days are also going to try for the Guinness world record for the most software downloaded in 24 hours.

I’ll certainly be one of the downloaders, it’s a much better browser than IE. If you feel like committing to the download click the image to register your pledge.

There were many pictures that Panthers wanted to use and these have been displayed using a “hoverbox” effect and also incorporated some FlickR slideshows into the training pages. The new website has been launched and is going through some final tweaks, and the club are very pleased with our efforts to provide what they wanted. Panthers are now working on getting themselves some good rankings for searches.

The point is that if there is no competition and, more importantly, no-one searches for crowdedalliances therefore it is simple to rank for the term. Yet many website owners will go for such words.

What should be of concern is what searches are made that your website could fulfill, what those keywords are, and how your content reflects that. Otherwise you too could rank highly for crowdedalliances that returns no searches.