With the huge number of postings on all the various forums, concerning this update, most people don't know where to start looking for information about the recent Google update. The following is an attempt to put down rationally (I hope) most of the information that is known and the (unproven) theories behind the update algo.
Starting on the 16th of November, a major shift in results was seen on Google. Veterans recognised that Google appeared to be doing a major update, not seen for many months, as reported first on WebMasterWorld
who named it "Florida", continuing the tradition of naming updates rather like hurricanes. In this case it was a hurricane! As was usual with many updates, there were moans and groans as people complained about their sites falling. Many people were unaffected (including us) but the symptoms of the sites being dropped were not usual. No penalties, such as PR0, seem to have been applied against pages that had fallen - yet none of the pages targeted at specific key phrases, particularly index/home pages, appeared in the top results for these search terms. Indeed some had dropped hundreds of places and, in some cases reported, off the scale. Yet these pages did appear for obscure phrases and were obviously still in the index.
It appeared to us and to several other respected names (though hotly disputed by others) that some sort of over-SEOd filter had been applied to check if overt SEO had been done for that particular phrase. It was as if Google were checking to see if external links to the site included the phrase, on-page optimisation was being done for the phrase and even if the domain included the phrase. If the density of the optimisation, both on and off the page, appeared too artificial, then a filter was tripped and down went the page - solely for that phrase.
Google had never looked favourably on abuse of their systems and many established SEOs looked upon this algo tweak as a way of Google getting rid of the abuses of links and stopping the scrambling for getting (and sometimes buying) links including your required anchor text from other high PR, but probably irrelevant to your subject, sites. It seemed to make sense.
On Friday, 21st November, Google decided to tighten the "filter". All hell broke loose as tens of thousands of sites disappeared from positions they had held (in some cases) for years. We noticed some of our client sites plummeting for their major key phrase from being #1 to total invisibility. Yet this was only in highly competitive areas, not for their secondary phrases. These sites were, in most cases, not highly optimised, had not sought reciprocal links but had achieved their rankings through being on the web for 4 or 5 years. The bad news was that their company name and domain included the key phrase, sites (including directories) linking to those sites included the key phrase in their links and Google interpreted this as over-optimisation and down they plunged. In many areas all the top 20 ranking sites disappeared, including industry leaders, to be replaced by educational sites, news review sites, government sites, major shopping portals or directories. Something major had happened - but what?
Thousands of web pages have been suddenly demoted in the Google search results, primarily on the main commercial search terms for which they targeted their pages to be replaced by other sites who, in the main, referred to the search term obliquely. Several were the main shopping portals or business directories which gave listings for companies who may provide the services requested, many were not.
Very high-ranking "authority" sites seemed to be unfiltered.
The changes were starkly obvious on regional English language Googles where a regional filter was employed and there were less commercial sites with "authority".
An example for Google UK is the search for the word "shelving". On the UK search
the result used to show mainly UK suppliers of shelving. The results now show:
- #1 - about arranging shelving in Sheffield University - an education site .ac.uk
- #2 - shopping directory - Dealtime - a top commercial shopping site.
- #3 - shopping directory - Dealtime - a top commercial shopping site.
- #4 - Leeds University about needing shelving for their books - an education site .ac.uk
- #5 - Buckinghamshire College with an article about concerns with some shelving - an education site .ac.uk
- #6 - Kelly Search a UK Business Directory (like Yahoo or Yellow Pages) - authority directory site.
- #7 - Sheffield University library shelving sequences - an education site .ac.uk
- #8 - Article from a news site regarding the shelving of an Indian ferry service - news site.
- #9 - The same as 8 - news site.
- #10 - Information about the Loughborough University library shelving team - an education site .ac.uk.
On the main Google search for the same phrase
, the results return 1 site that sells shelving, 6 shopping portals, 2 Universities and 1 Amazon store. Yet previously these results showed 9 shelving suppliers.
Other searches remain unchanged. Take the word "jackets", for example. There could be plenty of diverse sites to come up for that phrase but, when searched for
- it returns the sort of results one normally got from Google. The new algo appears not to be being applied on this search.
WebMasterWorld members came up with an interesting observation which we confirmed. Only certain phrases were being selected for processing through the new algo. Indeed it was asserted that this could be proven by amending the search by asking to exclude a spurious string (such as -waffle) and it was true! If you do the search shelving -waffle
in the UK, all the sites that used to previously be listed in the top positions suddenly re-appeared. Similarly, do that search on the main Google with the same search
and all the shopping portals disappear and shelving suppliers re-appear! If you do the same thing with the phrase "jackets"
there is no real difference between the results previously shown. Why?
After tests on over 500 phrases we have concluded that certain phrases kick in the new algo, and certain don't. The ones that do are always highly commercial and the ones that are not, are often also commercial. So why some and not others? Completely non-commercial phrases never come up with any indication that the new algo/filter is being applied.
If you try this yourself and your phrase is more than 2 words you may have to add additional garbage words so web design Toronto may need to be web design Toronto -waffle -woggle.
These are the facts, though, it must be said that it is unlikely for Google to continue allowing this particular loop-hole in to viewing the different results, if they apply, for very long - so you may not continue to see the differences in a few days. If you don't, they were there!
Many theories abound as to what has happened and are hotly disputed. We have our own which are from some considered research and are included in this analysis and will be labelled as such. Others are available and will also be included with our comments. Be aware these are theories, some are bizarre, some may be close to the truth - but only Google knows what they are doing. They are here to assist you in your own conclusions.
The most logical theory
in my opinion - portions of which are agreed, I think, by most industry observers.
Google has seen their search engine results manipulated by SEOs to a significant extent over the past few years. Their reliance on PageRank™ to grade the authority of pages has led to the wholesale trading and buying of links with the primary purpose of influencing rankings on Google rather than for natural linking reasons. In some instances, people would not link to sites unless they thought it would not harm them or would benefit them for Google. Google SEO consisted mainly of ensuring your page had your primary search phrase placed in all appropriate places on your page and that your sought after phrase was inserted in the anchor text of any incoming links. Thousands of SEOs and webmasters followed these simple and basic rules and loved the fact that Google seemed to reward them by giving them top listings. People forgot the fact that Google really wanted to give surfers the most "authoritive" results. So someone looking for cheap computers found the site that had "earned" a reputation for providing cheap computers - not just that a webmaster had optimised their site to make it look that way by getting links with the phrase "cheap computers" included on a load of unrelated sites. This update was an attempt to redress the balance and get back to the way Google thinks that results should be calculated. The most obvious way to start is to identify the major terms that are likely to be manipulated. These are obviously commercial and there are a number of ways this could be done:
- Google has an immense amount of information fed back to it by their popular toolbar that could give an indication of what people searched for when looking to buy.
- Google AdWords is a dictionary of commercial terms built by webmasters themselves coupled with the amount of money they are willing to pay per click. This could indicate, very simply, the largest volume searched for terms and their "sought-after" nature.
- Forms of artificial intelligence and semantic analysis through programs used by Google. It could be worth reading the Applied Semantics White Paper - used by AdSense and also the Hilltop algo - both capable of being used by Google.
Which ever method is used (it may be none of the above or combinations of all of them), Google manages to identify sought-after search terms. It then, when the search term is looked for, applies an optimisation filter to redress the influence SEOd sites may have and reveals the adjusted, "authoritative" results. This is why you tend to see governmental, educational, shopping portal, business directory and news sites on many searches. However, some commercial sites will still be considered the authority on that subject and remain - many, though, will drop out. This is the theory we subscribe to. We believe this was, and is, the intention of Google in this update.
Additions or replacements to this theory include:
- Google is trying to force webmasters who have been seen by Google as manipulating their results to use AdWords and that is why they have removed the results. My conclusion - this may be a welcome side benefit of the change (for Google), but not the primary reason for it.
- Google is trying to separate their results into non-commercial and commercial. Commercial results will be AdWords and non-commercial the natural listing, primarily used for research purposes. My conclusion - well, it was probably me who started this, with a rather cynical tongue-in-cheek remark on the High Rankings Forum. Well, in reality, this is pretty much impossible and would defeat Google's idea of returning the most suitable sites for all searches. But some searches do make it look that way. However, in my opinion, this is not true.
- Google has changed their algo to suit their biggest advertisers. My conclusion - highly unlikely and I do not believe that Google has any wish to be seen as an unethical company, regardless of their growing unpopularity with "jilted" webmasters.
- There is no separate algo for "commercial" phrases and this is a new over-all update. My conclusion - all evidence points to this not being the case. I'm waiting to be convinced I am wrong, though.
- This is a result of a virus or blog results upsetting the standard results which can only be seen if you type in -mt-tb.cgi as revealed in The Register. My conclusion - this is confusing the results returned unfiltered as shown with our -waffle (or any other garbage word) with a method of getting rid of extraneous listings from Blogs in the search results by excluding from the search the signature of the most used blogger trackback script.
Despite the furore, there is no great conspiracy theory. Google are trying to redress the balance by trying to eradicate obvious search engine manipulation from their most competitive results. Remember, it is what Google considers their most competitive results probably gauged by both value and number of searches. As such, certain types of site don't fall into this category. SEO sites, for example, may be competitive but are not searched for with huge frequency.
The results of the filter have probably surprised Google, but, for the moment, few surfers will notice anything - unlike webmasters. The quality of the results has changed, with searches for home alarm
, web design Calgary
, and medela breast pumps
showing, variously, sites with nothing to do with home alarms, a hockey team being top for web design and a listing of pretty much nothing but shopping portals. (My thanks to some of the many people who have posted some of the searches they have seen and I have used - you know who you are!) Many feel this has been a severe downgrading in relevancy, others believe that this is an improvement in the surfer experience. It is your judgement!
It is certain that Google will be watching the result of their change and will adjust the algo if they think it will reward sites of merit and increase relevancy. Whatever happens, though, the days of easy rankings through simple SEO are over.
As always, any comments on this article, designed to be a basis for informed discussion on the latest Google update will be welcomed. All opinions are the author's and not necessarily those of the Company.
November 25, 2003