This week, we're taking a look at a patent application that explains how search engines detect spam. Do search engines think that your website is spam?

In the news: Google tests a "skip intro" link for websites with a welcome page, Google shows Sitelinks for sub-directories, Yahoo might still be important and more.

Table of contents:

• Facts of the week
• Search engine news and articles of the week
• Success stories
• Previous articles

We hope that you enjoy this newsletter and that it helps you to get more out of your website. Please pass this newsletter on to your friends.

Best regards,

About three weeks ago, Microsoft was granted a new patent with the name Web Spam Classification Using Query Dependent Data. Although this patent application was filed by Microsoft, all major search engines probably use similar methods to classify web pages.

How do search engines analyze web pages?

Search engines look at a number of elements that can appear on web pages and within queries that web surfers use to find these pages.

For example, search engines may look for the most frequent keyword in the web page, the number of times a particular keyword appears in the web page, the domain name associated with the web page, the number of links pointing to the page, the HTML tags in which a keyword appears and many other factors.

The patent filing indicates that search engines look at hundreds of different factors to rank web pages.

How search engines try to detect spammy pages

The are so many potential spam pages on the Internet that search engines cannot identify all spam pages manually.

To identify potential spam pages, search engines might manually label some web pages as spam and then take information from that pages to find other spam pages.

For example, a web page that uses keyword stuffing has more keywords than a legitimate page. By training the spam detection algorithm with a few web pages that use keyword stuffing, other web pages that use keyword stuffing can be detected automatically.

In other words, a spam detection algorithm labels web pages as spam or not spam by looking at decisions made by humans. According to the patent application, the algorithm might look at the following factors:

• the number of inbound links coming from labeled spam pages
• the top level domain of the site
• the quality of phrases in the document and density of keywords (spammy terms)
• the count of the most frequent term
• the count of the number of unique terms
• the total number of terms and the number of words in the path
• the number of words in the title
• the rank of the domain and the average number of words
• the top-level domain
• the number of hits within a domain
• the number of users of a domain
• the number of hits on a URL and the number of users of a URL
• the date the URL was crawled, the last date page changed
• many more factors

If your website uses similar elements as the spammy web page then it's likely that your website will be classified as spam. The usual impact of a website being labeled as spam is that the site might be pushed down in search results, or removed completely.

What does this mean for your website?

You should make sure that your web pages use similar elements as the top ranked pages instead of elements that can be found on spam pages.

The easiest way to make sure that your web pages use all the elements that are used by top ranked pages is to analyze your pages with IBP's Top 10 Optimizer. IBP will analyze all important elements of your web pages and it will tell you in plain English what you have to do so that your web pages get top 10 rankings.

If search engines label your website as spam you will lose a lot of visitors and customers. You should analyze all elements of your web pages to make sure that search engines label your web pages as high quality content and not as spam.

Back to table of contents - Visit Axandra.com