Home Services Contact Info

Vodafone.ie – Problems With Robots META Tag

Posted in: Search Engine Optimisation by Richard Hearne on February 9, 2009
Internet Marketing Ireland

Have you ever noticed a spike in daily traffic, say of the magnitude of 100-200%? It’s the sort of thing you’re unlikely to miss (unless you don’t watch your stats, but of course you watch your stats, right?). Sometimes the cause of that spike has nothing to do with your site at all. Sometimes it has much more to do with problems other sites are having. And the good news? You can very often learn from other’s mistakes to your advantage.

Be Careful of Robots META Tag

You can include a META tag with your HTML that contains instructions for Search Engine Crawlers. It looks like this:

<meta content="noindex,nofollow" name="ROBOTS" />

Here’s what happens when you include the above instruction in visual terms:

Vodafone.ie Homepage with NoFollow Highlighted
Vodafone.ie Homepage with NoFollow links Highlighted in Pink

Not only is the page de-indexed, but also all links on the page are set to NoFollow – they cease to exist in the eyes of the Search Engines. The Robots META is like the big brother of the rel attribute on HTML elements. While rel applies only to the element it is an attribute of, the Robots META is applied to the entire page it appears on. Therefore it’s far more blunt than rel, and also far more powerful.

In fact the META tag I block-quoted above is the actual Robots META tag from www.vodafone.ie (I wrote previously about Vodafone.ie homepage here). I first noticed this last Thursday, and all images etc. were taken last Saturday Feb 7 by the way.

How Google Reacts to Robots=”noindex,nofollow”

Here’s the outcome once Google sees this tag on your pages:

Google Search Result for [Vodafone.ie]
Google Search Result for [vodafone.ie]

You can clearly see that WWW is no longer in the results, except for some search results pages (which Google doesn’t want sites to include BTW) which appear to be sitting on a GSA (probably the reason they don’t have the same Robots META).

Moral Of The Story?

Be extremely careful with the Robots META tag. You should occasionally check your site, especially your homepage, to ensure nothing has gone awry. You should also check your analytics regularly, I would say at least daily for large corporate sites that run e-commerce modules, as you’ll likely see issues such as this instantly.

How did I notice this?

Well I login to GA each morning and spend about 30-40 minutes digging through the stats for redcardinal.ie and all other client sites I have access to. When you’re traffic jumps 2.5x over the past few days it tends to make you dig a little deeper. I imagine the guys in Vodafone’s mobile email dept might also be wondering why Google is sending them so much traffic.

[Sidenote: Interesting to see Vodafone.ie are using Woopra rather than GA now.]

How To Avoid This?

You can easily avoid this issue, and some of the solutions are quite simple:

  • Build some sanity-checking into your CMS. Make it alert you when the Robots tag contains “noidex”.
  • Utilise a diagnosis product, such as Maximine, to alert you when it finds this META tag.
  • Build a simple manual SEO-checkpoint into your QA process. You could simply use a plugin to highlight NoFollow (as I have) which would instantly show the issue during QA.
  • Check your Search Engine referral logs daily to watch for spikes (negative and positive).
  • Check your Google Webmaster Console for data and stats. In particular Statistics->Crawl Stats contains some very useful info for both marketing and network engineering folk. (NB: make sure you set up accounts for all relevant domains and subdomains.)
  • Create a Google Alert for “site:[mydomain.tld]” and add it to your feed reader. (I haven’t tested this yet.)

I’d love to hear if you have any other ideas for checking that your site is indexed by Google, or if you know of any service offering this ability? Oh, and is anyone else using Woopra – any feedback to offer?

You should subscribe to the RSS Feed here for updates.
Or subscribe to Email Updates now:

7 Comments »

  1. Hi Richard, it’s not just the ‘back end’.
    I was literally on the Vodafone website 5 minutes ago thinking ‘this site is a real mess’. Visually it’s all over the place. I know over the years they have had some really great designs presented to them and it’s still an awful site to use and experience. What gives?

    Comment by Peter Knight — February 9, 2009 @ 9:32 pm

  2. Hey Peter

    Another really terrible thing is that their SSL cert is only valid for http://www.vodafone.ie, yet they allow their site to resolve on vodafone.ie as well. For an e-commerce site this is akin to throwing away money, as many users get popups telling them the site is insecure.

    Can’t really give an opinion on the design – I’m more neutral on that TBH. IA is quite poor though, that’s for sure.

    You’d almost be forgiven for thinking the mobile telcos don’t care about their customers…

    Rgds, and thanks for dropping by
    Richard

    Comment by Richard Hearne — February 10, 2009 @ 5:31 am

  3. That looks like a bit of a slipup, might also add to their PPC bill. Surprised nobody noticed this.

    Comment by Michael — February 10, 2009 @ 2:02 pm

  4. ouch !

    One handy tool is http://yellifitchanges.com/google_serp.php if don’t have the expertise to get someone to program something for you. It will tell you when the SERPs change for a particular keyword, and you should hope that querying your domain name is something that you will rank for.

    yellifitchanges was written by someone from Cork.

    Paul

    Comment by paul savage — February 11, 2009 @ 3:34 am

  5. Hey Paul

    Interesting tool. Giving some odd ranking results, so I wouldn’t rely on it for that.
    But for monitoring your rank for your domain should be useful.

    Do you know who created it, and if so maybe ping me their details pls?
    Rgds
    Richard

    Comment by Richard Hearne — February 11, 2009 @ 4:00 am

  6. Done Richard ! Took a little investigating, but it’s possible to find the lad.
    Paul

    Comment by paul savage — February 11, 2009 @ 9:08 am

  7. Nice post as per usual. I have tried using robots.txt/nofollow to get a hold on poor CMS design before. It can be a little daunting. I think the announcement by the big 3 engines on the new tag to deal with duplicate content is sure going to help with this. I am hoping to get started with this early tomorrow morning ..

    Comment by Kieran — February 15, 2009 @ 9:09 pm

Comments Feed TrackBack

Leave a comment