10 Jan 2006

What does Google know about your domain names? by Nick Wilsdon

Last month I asked Nick Wilsdon if he’d write an article about, "What does Google know about your domain names?" and today he responded with




That article on TW today prompted me to finish this – http://www.threadwatch.org/node/5238 – I think my notes went further than that.

 

So here ya go, my first guest article is a great one, and it’s by Nick Wilsdon. (Nick Wilsdon has worked as a domain registrar for Tucows since 2000, as well as being an active member of their beta development team. He founded e3internet.com; a web production group based in Russia and is a regular contributor to Multilingual-Search.com.)

What does Google know about your domain names?

The answer is probably a lot more than you think they do. I’ve been prompted by Jim and several recent posts in the SEO community to try and answer this question. I’ll apologise now if you find some of this material simple. Largely it is once explained. However subjects touching on domain registration always seem to be surrounded by quite a lot of FUD. Hopefully this article can clear some of that.

One such issue was Google becoming a registrar back in February of 2005. To understand this move we first have to understand what exactly it means to become a ‘Registrar’, what benefits does it give you and how can it be utilised by a company like Google?

Domain registration is very much a business of resellers. Basically anyone who sells domain names is a domain registrar but they may be 2, 3 or 4 tiers down from the source, the Registry in charge of that domain extension. For example in the case of the .com extension Verisign was put in overall management authority. In most cases being a reseller is simply more practical. It costs a few thousand dollars to sign up for each individual Top Level Domain (TLD) or Country Coded Top Level Domain (ccTLD), reselling for a larger Registrar such as GoDaddy or Tucows gives you access to their entire portfolio of domain extensions.

However it seems that Google did not start signing as a Registrar in order to buy or sell domains; they did this to have greater access to domain information. Now to clear some of the FUD that was speculated about this. Google as, say a .com Registrar, does not have access to all the customer records of Verisign. They can only access the further details of domains which they personally sold. Tucows and GoDaddy are both Registrars with Verisign, do you think they have access to each other’s entire customer database? Certainly that information would be well worth the $3500 fee. No. Unless the domain is within their own account they have exactly the same access as you or me using the public WHOIS records.

Well that’s not entirely true. There is one very important benefit to becoming a direct Registrar with each ccTLD and TLD Registry; access to their APIs. While the Registries do provide a public WHOIS service this is often limited. In much the same way Google and Yahoo restrict automated searches, the Registries restrict queries on their data. As a Registrar you are entitled to carry out high volume automated queries.

I’m sure at this point several of you have already reached my conclusions. I believe Google has built or is building a tool to analyse domain names. The API access they were given as a Registrar allows them to carry out the level of automated queries they needed for this. I would also go further and suggest this tool is building up a historical picture of each domain through regular scraping of their WHOIS records.

As you can imagine this tool could be very useful for Google. It would allow them to track the age of domain names and connect likely networks and portfolios. They would see when a domain name changed ownership and allow them to adjust the value of that domain in their engine. As Jim wrote some time back, we have all learnt to appreciate old domains; they escape the aging filters as well as saving you months if not years of link building work. It’s considered good practice when buying an old domain name not to change it radically, adding hundreds of new pages suddenly or overhauling the design but what about changes in the WHOIS?

Ah yes, but we can cover our tracks here with Domain Privacy. Or can we? As I have explained before, Google can not access the records of other clients of the Registry. However their tool could be used to flag domains which take out this service. It is also important note that several pieces of information are not included in domain privacy services. These are the current Registrar, record creation date, expiry date, renewal date and the name servers associated with the domain. A system could be set to look for one or more changes in this data as an indication the domain may have changed hands. Changing too much of this data plus a domain privacy service appearing on a domain might raise a flag or reduce Google’s trust in the domain.

They would especially be watching domain which are about to drop (expire) as this would be the likely time they would change hand. The system would be looking for status changes on WHOIS records, such as DETAGGED, REDEMPTION PERIOD, or PENDING DELETE. These would all indicate the registrant had failed to renew their domain. None would be hidden by domain privacy contracts.

This article is speculation. I think it is considered speculation but only Google knows. This theory would go someway to explaining the success that some marketers are still having on some expired ccTLD’s; it is unlikely that Google has integrated these into their system as yet. While they may be “wiping the slate clean” on expired .com domains, they don’t have this level of data from these other Registries. Some, such as the Russian (.ru) Registry have very limited API capacity at the moment which might be a barrier to inclusion in such a system.   

It is certainly Google’s modus operandi to accumulate information and incorporate that data into their algorithm. As Eileen Rodriguez, a Google spokeswoman wrote "While we have no plans to register domains at this time, we believe this information can help us increase the quality of our search results." The Google Patent increased speculation on positive benefits of changes in the WHOIS, notably increasing the registration period, but little has been written on negative effects that some changes could cause. Carefully management of this information may have just become another string the SEO/SEM has to add to their bow.    

——-
Added by Jim:
Nick offers Domain Name Registrations , e-commerce hosting, as well as Internet Marketing Strategies. More on Nick

Great Artilce Nick! Thanks for letting me publish this on my blog.