Is real google bot like "fetch"or more like "fetch & render"?
-
In GWT we have two options to mimic googlebot visits, "fetch" and "fetch and render", but when the real googlebot visit a page, is he behaving like the former or the latter?
I can see fetch does fetch only the html, while fetch and render does fetch .js and .css as well.
But what does the real googlebot does?
I have checked the web server logs, and I can see the real googlebot sometimes request the .js files too, but not every time it visit a page, sometimes it does, sometimes it does not.
Has anyone figured out when googlebot actually request javascript files?
-
That was my guess, that google somehow cache the .js files, so I changed one javascript file name to a random generated name. Did it last night, I will recheck logs in a couple of days to see what happen.
-
Googlebot is behaving like the latter, "Googlebot constantly crawls and renders URLs across the entire web..." from https://support.google.com/webmasters/answer/6066467, but has the option of streamlining the process by not rendering every time. For example, if many sites are referencing the same centrally hosted code base of js, googlebot doesn't need to constantly recheck that js every time it comes across the link.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Search Console - Click Count Inconsistency
In Google's search console I see a discrepancy between click counts. At top I see this here and then beneath I see these kinds of numbers for click counts here. So the top click count says 252 and the bottom section appears to only shows less than 40. Probably a simple explanation here that I'm just not seeing. Thanks!
Reporting & Analytics | | a_toohill0 -
Google Crawl Stats
Hi all Wondering if anyone could help me out here. I am seeing massive variations in WMT of google crawl stats on a site I run. Just wondering if this is normal (see attached image). The site is an eccommerce site and gets a handful of new products added every couple of weeks. The total no of products is about 220k so this is only a very small %. I notice in WMT I have an amber warning under Server connectivity. About 10 days back I had warnings under DNS, Server and Robots. This was due to bad server performance. I have since moved to a new server and the other two warnings have gone back to green. I expect the Server connectivity one to update any day now. Ive included the graph for this incase it is relevant here. Many thanks for assistance. Carl crawlstats.png connect.png
Reporting & Analytics | | daedriccarl0 -
What about this (google crawl)?
Recently we did a serious effort on SEO with SEO Yoast (Wordpress). And after a few months of tweaking old articles we get this impact on crawl search.. Is this graph normal? s1TQgv9.png
Reporting & Analytics | | noodweerbenelux0 -
Google Analytics Goals - Different Domain
Hi, We currently need to setup a goal that involves a different domain. Any guidance would be greatly appreciated. more info: the checkout process is redirected to a secure.domain Just let me know if I can provide any additional info that may helpful.
Reporting & Analytics | | opusvo0 -
Why does Google dislike us so?
Hi all, Over the past 18 months one of the sites I look after, www.english-inns.co.uk, has taken a massive dip in Google rankings (as per the graph attached). To give you an idea of what the site is about, we're effectively the largest directory of inns in England and have a mixture of exclusive offers direct from member inns but also have some affiliate links thrown into the mix. In terms of steps taken to remedy our falling out of grace with Google, these have included: A complete website redesign Removal of any duplicate content 'Adding value' by including interactive maps, better search functionality, promoting exclusive offers, etc. Unfortunately none of this seems to have helped our situation and we are now sitting at some pretty horrible places in Google's SERP's. For example, even for the keyphrase "english inns" we only rank #5 and this is a term associate with our brand! Perhaps I'm missing something obvious here... can anyone see any glaring issues with the site? Any help would be most appreciated Mozzers. Cheers google-analytics.png
Reporting & Analytics | | dooberry0 -
Google Analytics - Referral Traffic Question
How Google Analytics determine that some particular web site referred traffic if there is no back link on that site?
Reporting & Analytics | | DiamondJewelryEmpire1 -
My rankings are up and down like a Yo-Yo. Any ideas why?
One of the companies that I work for sells the latest smartphones. It ranks fairly well for key terms in an extremely competitive field. Recently the rankings of one particular handset have been extremely erratic and do not follow the pattern of other products, I am wondering if anyone can offer some insight as to why this is happening? When the handset was first announced, it ranked fairly well (20th). This is good when you bare in mind that it is an ecommerce site competing with top tech sites. Prior to release, the ranking gradually declined to 33rd. I believe this may be due to reviews being publsihed (as a result of review handsets being sent out prior to release) as Google has started to favour reviews and news updates. I have seen a similar pattern with other handsets, reviews and news are favoured by Google prior to release, but once the product is available, my rankings rise and remain on the first page provided that links are still coming in. However, the rankings of this handset dropped from 27th to below 50th upon launch. The next week the ranking rose from >50th to 7th, dropped to below 50th again the following week, rose back to 9th the week after and has now dropped again to below 50th. These are by far the biggest changes in rankings I have seen on a week to week basis. The page has a strong link profile in order to achieve such a high ranking in the first place. With this being the case I have no idea why ranking can drop off so much in the space of a week? Update: I forgot to mention initially that I have the variant listed twice, one for each colour variant. All content is the same apart from the colour being altered in the appropriate places. This has not been a problem in the past, but perhaps it is due to recent changes to Google algorithm?
Reporting & Analytics | | pugh0 -
Google analytics - help
Hello, What is meant by campaign expiration in Google analytics ? The default value is 6 months. what does it mean ?
Reporting & Analytics | | seoug_20050