Found us from search engine?
We rank high, you can too.
SEOPressor helps you to optimize your on-page SEO for higher & improved search ranking.
If you don’t want to spend 40 minutes watching the recording, here’s a quick summary of the important key points of the session.
A brief background introduction on the presenters. Tom Greenaway is a senior developer advocate from Australia. While John Mueller (aka johnmu, ring a bell?), is Google’s webmaster trends analyst from Zurich, Switzerland.
Tom started the talk by sharing a little background of search engines. The purpose of search engines is to provide a relevant list to answer user’s queries. A library of web pages is compiled where answers are pulled from. That library is the index.
Building an index starts with a crawlable URL. Now, the crawler is designed to find contents to crawl. To do this, the content must be retrievable via an URL. When the crawler gets to an URL, it will look through the HTML to index the page as well as find new links to crawl.
Here’s what you need to know, Tom shared the six steps to ensure your web page will be indexed.
2.Utilize canonical tags
– In case of content syndication where a content is distributed on different sites to maximize exposure. The source document should be tagged as the canonical document.
3. Make sure the URL is clean and unique
– Don’t list session information on the URL.
4.Provide a sitemap to Googlebot
– That way the crawler has a list of URLs to crawl and you can sleep better at night knowing your website is properly crawled.
5. Use history API
– Which replaces the hashbang tag(#!), which, if used will no longer be indexed.
6. Make sure your links have anchor tags with HREF attributes
– Googlebot only recognizes links with BOTH anchor tags and HREF attributes, otherwise, they won’t be crawled therefore never indexed.
which, you should take a look at so you won’t make the same mistakes.
2. Lazy loading images
– They are only sometimes indexable. To make sure that they are properly indexed,use noscript tag or structured data.
– Take caution, images only referenced through CSS are not indexed.
3. Any contents that are triggered via an interaction won’t be indexed
-Googlebot is not an interactive bot, which means he won’t go around clicking tabs on your website. To make sure he can get to all your stuff either preload the content or CSS toggle visibility on and off.
– What’s better, just use separate URLs to navigate user and Googlebot to those pages individually.
4. Rendering timeout
– Make sure your page is efficient and performant by limiting the number of embedded resources and avoid artificial delays such as time interstitials.
5. API that store local information is not supported by Googlebot.
– Instead, it crawls and renders your page in a stateless way.
– Rendering web pages is a resource heavy process, therefore rendering will be delayed for a few days until Google has free resources.
3. Two-phase indexing
– First indexing happens before the rendering process is complete after final render arrives there will be a second indexing.
– The second indexing doesn’t check for canonical tag so the initially rendered version needs to include the canonical link, or else Googlebot will miss it altogether.
– Due to the nature of two-phase indexing, the indexability, metadata, canonical tags and HTTP codes of your web pages could be affected.
John Mueller takes the baton and shares with us some basic information on
2. Server side rendering
– Your server deals with the rendering and serve users and search engine alike static HTML.
3. Hybrid rendering (the long-term recommendation)
4. Dynamic rendering (the policy change)
– This method sends client side rendered contents to users while search engines got server side rendered content.
– This works in the way that your site dynamically detects whether its a search engine crawler request.
– Device focused contents need to be served accordingly (desktop version for the desktop crawler and mobile version for the mobile crawler).
Now that it is out in the open that Google prefers the (NEW) dynamic rendering method to help the crawling, rendering and indexing of your site.
– Could be run as a software or a service that renders and caches your content on your side.
Both of these are open source projects where customization is abundant. John also advises that rendering is resource extensive, so do it out of band from your normal web server and implement caching where needed.
The most important key point of dynamic rendering is the ability to recognize a search engine request from a normal user request. So how could you recognize a Googlebot request? The first way is to find Googlebot in the user-agent string. While the second way is to do a reverse DNS lookup.
John stresses during the session that implementing the suggested rendering methods is not a requirement for indexing. What it does, is making the process easier for Googlebot. Considering the resource needed to run server side rendering, you might want to consider the toll before implementing.
Now let’s look at when you don’t need to use dynamic rendering. The answer is simple, if Googlebot can index your pages correctly, you don’t need to implement anything. So how can you know whether Googlebot is doing their job correctly? You can employ a progressive checking. Keep in mind that you don’t need to run tests on every single web pages. Just test perhaps two each from a template, just to make sure they are working fine.
2. Run a Google Mobile Friendly Test. Why? Because of the mobile-first indexing that is being rolled out by Google where mobile pages will be the primary focus of indexing. If the pages render well in the test, it means Googlebot can render your page for Search
3. Keep an eye out for the new function in the mobile friendly test. It shows you the Googlebot rendered version and full information on landing issue in case it doesn’t render properly.
4. You can always check the developer console when your page fails in a browser. In developer console, you can access the console log when Googlebot tries to render something. Which allows you to check for a bunch of issues.
5. All the diagnostics can also be run in the rich results test for desktop version sites.
At the end of the session, John also mentions some changes that will happen. The first happy news, Google will be moving rendering closer to crawling and indexing. Which I assume will mean that the second indexing will happen much quicker than before. The second and last happy news, Google will make Googlebot use a more modern version of Chrome. Which means a wider support of APIs. They do make it clear that these changes will not happen until at least the end of the year.
With that, the session is concluded. Do check out our slide show for a quick refresh. All in all, Google is taking the mic and telling you exactly what they want. Better take some note.
Updated: 21 June 2018