Comments on: After more digging I found this: https://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/ Search Engine Optimisation Ireland Mon, 31 Aug 2015 22:05:56 +0100 hourly 1 http://wordpress.org/?v=3.9.1 By: John Muellerhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1558012 Thu, 05 Feb 2015 10:27:52 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1558012 I don't think there's currently a way to determine that absolutely (apart from things that are obviously blocked, or directly included in the static HTML). 

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557284 Wed, 04 Feb 2015 16:05:22 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557284 Thanks John

That's really very useful.

So now my last question for you. If you visited a website such as http://www.hubspot.com you'll see (on desktop) a navigation menu at top of page.  All fine and normal. Until you view the source and you'll only find the scaffold template based in script tags.  Google's cache shows the rendered nav, BUT the text only cache shows a nav contained in a <noscript> element.

In the good old days JS navs were just bad.  Those were black and white days.  Now how we can tell whether an implementation such as above is OK from a search perspective?  Is there any easy way to determine if Google really is counting these exotic navigation links?

Sorry for extended questioning, but I suspect this will be useful to many people.

]]>
By: John Muellerhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557283 Wed, 04 Feb 2015 15:57:36 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557283 This is all still fairly new, we've been working on indexing JS content for a while, and we're getting pretty good at it, but I wouldn't say that all JS content is already 1:1 equivalent to static HTML. There are lots of moving parts, especially on the website-side. Where we can recognize the links in the rendered version, we'll try to treat them as equivalents to static HTML links (passing the same signals, anchor text, etc). I'd like to say that we'll treat them exactly the same all the time, but I can imagine there may be abuse scenarios where we have to do something different. 

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557282 Wed, 04 Feb 2015 15:42:18 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557282 I hate to ask you John, but the "more or less" throws me, so apart from latency will the links be treated similarly i.e. will they pass the same signals as a plain HTML link?  If we build our navigation via a JS render, which Googlebot also renders, will it have the same outcome as a HTML rendered equivilant?

The other big issue I'm having is determining if things like navigation flyouts (hidden until an event happens) rendered via e.g. Handlebars are getting rendered on not by Googlebot. Since they're hidden from initial view we cant tell from the GWT Fetch and Render tool, and the Fetching data view simply shows the scaffold inside script tags.

Sorry to ask, but there seems to be a serious lack of testing/discussion around some of this :(

]]>
By: John Muellerhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557281 Wed, 04 Feb 2015 15:36:11 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1557281 +Richard Hearne yes, if we discover links through rendering, we'll treat them (more or less — there's an element of latency in there) the same as other links on the page. 

]]>
By: Trey Collierhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556962 Wed, 04 Feb 2015 07:44:47 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556962 If you were wondering what a SPA was/is…..
http://en.wikipedia.org/wiki/Single-page_application

I'm a noob….had to do some googling to catch up here!  :P

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556828 Wed, 04 Feb 2015 03:33:00 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556828 +John Mueller sorry it's my bad for bringing that particular SPA implementation into the discussion. What I'm really interested in is when JavaScript rendered content Google is indexing.

My use-case is that I'm working with a very big blog which renders comments via JS. The comments used NOFOLLOW for links, but I've found that some people found a loophole that allowed them to get FOLLOW links in the comment bodies. I asked +Pierre Far and he said we could add NOFOOLOW to any link we have concerns about, but If I have a concrete justification that links rendered by JS may be interpreted the same as HTML-rendered links it will be far easier for me to sell this to stakeholders.

Any help here is much appreciated.

]]>
By: John Muellerhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556617 Tue, 03 Feb 2015 21:41:47 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556617 These things are bound to change over time, so I'd be wary of reading too much into this specific implementation :). The idea is not to make your lives harder – really! -, but rather to make sure that we can show your site for all of the relevant queries that it deserves, based on the content that users would see. 

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556467 Tue, 03 Feb 2015 17:01:16 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556467 Thanks, nice find.  But I suspect the key to this is that it wont do any XHR after the $(document).ready state change.  Even that seems to be contradicted by other tests.

On Disqus – if you look at the source code for any seland post you'll see the comments rendered in HTML.  FB is actually rendering in an iframe, but the iframe is generated by XFBML as opposed to sitting in the rendered page.

Part of me thinks that what they announced in 2011 may have been a little bit of vapor TBH.

]]>
By: Edwin Jonkhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556466 Tue, 03 Feb 2015 16:54:53 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556466 As far as I understand both Disqus and FB use iframes that are generated by js. Following my examples, the iframe points:

For disqus to
https://disqus.com/embed/comments/?base=default&version=ff15479433461993d0738de53d5f22cf&f=searchengineland&t_i=85686%20http%3A%2F%2Fsearchengineland.com%2F%3Fp%3D85686&t_u=http%3A%2F%2Fsearchengineland.com%2F1-for-the-in-house-search-engine-marketer-85686&t_e=%2B1%20For%20The%20In-House%20Search%20Engine%20Marketer&t_d=%2B1%20For%20The%20In-House%20Search%20Engine%20Marketer&t_t=%2B1%20For%20The%20In-House%20Search%20Engine%20Marketer&s_o=default&l=#2

For FB to
https://www.facebook.com/plugins/comments.php?api_key=187288694643718&channel_url=http%3A%2F%2Fstatic.ak.facebook.com%2Fconnect%2Fxd_arbiter%2FDU1Ia251o0y.js%3Fversion%3D41%23cb%3Df1f3a7654c%26domain%3Dtechcrunch.com%26origin%3Dhttp%253A%252F%252Ftechcrunch.com%252Ff203a0b4f8%26relation%3Dparent.parent&href=http%3A%2F%2Ftechcrunch.com%2F2011%2F04%2F12%2Ffacebook-comments-now-on-over-50k-sites-get-more-social-with-latest-upgrade%2F&locale=en_US&numposts=25&sdk=joey&width=100%25

]]>
By: Edwin Jonkhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556465 Tue, 03 Feb 2015 16:53:27 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556465 I understand the HTML snapshot. After some digging I am pretty sure Googlebot can do XHR requests [1].

we improve both the indexing of a page and its Instant Preview by following the automatic XMLHttpRequest generated as the page renders.

However, Googlebot might only preform one request. And, thereby, the content/comments that are loaded "automatically" when you visit the page are not "seen" by Googlebot. 

http://googlewebmastercentral.blogspot.nl/2011/11/get-post-and-safely-surfacing-more-of.html

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556464 Tue, 03 Feb 2015 16:45:38 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556464 Quick follow-up on Facebook comments on Techcrunch:

<fb:comments href="http://techcrunch.com/2015/02/02/uber-opening-robotics-research-facility-in-pittsburgh-to-build-self-driving-cars/" num_posts="25" width="100%"></fb:comments>

So they are embedded using XFBML, and it looks like Google must have built a custom way of crawling this and parsing it to the page.  If you look at the source of Google's cached text version you'll see where they inject the crawled comments block.  Nasty looking stuff,

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556463 Tue, 03 Feb 2015 16:38:05 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556463 Hi Edwin.  Disqus can be loaded either via JS client-side, or an API server-side.  So if you look at what seroundtable was doing, they were loading the comments server-side into the HTML, but allowing the JS to also load client-side.  The HTML comments were for Google, and JS for users.  They turned off the server-side stuff after John Mueller told Barry his comments were causing quality issues.  

I haven't looked at Facebook comments, but I assume Google is doing something a bit custom to pull them in.  Especially if the comments appear in an iFrame (as in case of Techcrunch).  Thanks for the input though – more things to think about now.

]]>
By: Edwin Jonkhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556462 Tue, 03 Feb 2015 16:25:19 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556462 +Richard Hearne I think you are wrong here, namely both Disqus and FB comments can be indexed by Googlebot on the page. Just two examples.
Disqus comment on SE-land:
https://www.google.com/search?q="Do+you+think+this+is+a+side+stepping+attempt+to+incorporate+a+reward+for+advertising+on+Google+to+organic+listing+result+rank%3F"
FB comment on TechCrunch:
https://www.google.nl/search?q="Facebook+comment+box+is+awesome+and+if+you+know+how+to+put+some+extra+emoticons+in+your+comment+then+it+will+be+even+more+pleasent+:)"

My guess is that the link to iframe, which is somehow loaded via js, is being indexed and "merged" to the particular page, as if it is content on the page.

P.S. the example pages are from 2011, just to make sure they are indexed. Even though these are popular sites, it doesn't seem to be that the comments are indexed asap….

]]>
By: Richard Hearnehttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556426 Tue, 03 Feb 2015 15:51:54 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556426 Sorry, I should have given more context here +Eric Wu My interest is in whether links rendered to the page via JS and indexed by Google can pass PageRank and associated ranking signals.

The reason is that I'm working with a site that loads comments via JS, I suspect reasoning that these wont be indexed. The content isn't bad, but some users figures out how to get follow links in and I'm trying to find justification to get these NOFOLLOWed.  It's a very big site, and changes wont be made without such justification.

Any knowledge on this topic?  I know you've been working with SPAs.

]]>
By: Eric Wuhttps://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556425 Tue, 03 Feb 2015 15:47:33 +0000 http://www.redcardinal.ie/general/03-02-2015/after-more-digging-i-found-this/#comment-1556425 Not exactly the holy grail for SEO if you don't include all other major search engines, especially if you live in the states. Although Bing/hoo is a smaller marketshare, at scale I still wouldn't want to give up 20-30% of my traffic.

Also that doesn't include FB, Twitter, or anything else that needs to read structured data.

Serverside rendering on initial request is still the proper way to go for good crawlability / SEO.

]]>