How I can check if Google and other search engines will properly cache a page (a dynamic one)?
-
My site is currently disallowing search engine bots with the help of robots.txt. These dynamic pages can be crawled using Screamingfrog since they are linked to a static category page which is also linked to the homepage.
Thanks in advance!
-
Great idea using Screaming Frog first then :-
Then use Google Web Master Tools > Crawl > Fetch as Google to see what Google sees
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Non-indexed or indexed top hierarchy pages get high PageRank at Google?
Hi, We are creating some pages just to capture leads from blog-posts. We created few pages at top hierarchy like website.com/new-page/. I'm just wondering if these pages will take away more PageRank. Do we need to create these pages at low hierarchy like website.com/folder/new-page to avoid passing more PageRank? Is this is how PR distributed even now and it's same for indexed or non-indexed pages? Thanks
Algorithm Updates | | vtmoz0 -
Googles Search Intent – Plural & Singular KW’s
This is more of a ‘gripe’ than a question, but I would love to hear people’s views. Typically, when you search for a product using the singular and plural versions of the keyword Google delivers different SERPs. As an example, ‘leather handbag’ and ‘leather handbags’ return different results, but surely the search intent is exactly the same? You’d have thought Google was now clever enough to work this out. We tend to optimise our webpages for both the plural and singular variations of the KW’s, but see a mixed bag of results when analysing rankings. Is Google trying to force us to create a unique webpage for the singular version, and another unique webpage for the plural version? This would confuse the visitor, and make no sense.. the search intent is the same! How do you combat this problem? Many thanks in advance. Lee.
Algorithm Updates | | Webpresence0 -
Website and landing pages - Proportionate authority
Does website's (homepage) ranking going to influence landing pages ranking or vice-versa? If the homepage is ranking good for a "keyword", will that improve ranking of other landing pages which are optimised for related "keywords" & Vice-versa?
Algorithm Updates | | vtmoz0 -
Creating Content for Semantic search?
Need some good examples of semantic search friendly content. I have been doing a lot of reading on the subject, but have seen no real good examples of 'this is one way to structure it'. Lots of reading on the topic from an overall satellite perspective, but no clear cut examples I could find of "this is the way the pieces should be put together in a piece of content and this is the most affective ways to accomplish it". **What I know: ** -It needs to answer a question that precludes the 'keyword being used' -It needs to or should be connected to authorship for someone in that topic industry -It should incorporate various social media sources as reference to the topic -It should link out to authoritative resources on the topic -It should use some structured data markup Here is a great resource on the important semantic search pieces: http://www.seoskeptic.com/semantic-seo-making-shift-strings-things/ ,but I want to move past the research into creating the content that will make the connections needed to get the content to rank. I know Storify is an excellent medium to accomplish this off page, but only gives no follow attribution to the topic creator and links their in. I am not a coder, but a marketer and creating the backend markup will really take me out of my wheel house. I don't want to spend all of my time flailing with code when I should be creating compelling semantic content. Any helpful examples or resources welcome. Thanks in advance.
Algorithm Updates | | photoseo10 -
My Website No Longer Appears in Mobile Google Search but Does in Desktop...Why Is This?
For a long time my website has appeared in both desktop and mobile search in Google. Yet recently it has stopped appearing in mobile yet still on desktop. Any ideas why this is happening and how to rectify it please? Many Thanks.
Algorithm Updates | | WSIDW0 -
I can't understand why I am not rank one on SERPS
Hi Guys, I really cannot understand why I am no longer rank 1 on SERPs? My link data shows great weight in comparison to competitors, my on page SEO is good, nice and diverse on the alt text. I know there are a lot of factors that effect SERPs but I believe I have done well but am still not ranking? Have I missed something?
Algorithm Updates | | TomLondon
I really appreciate any thoughts and ideas. Thanks,
Tom0 -
Google and Wikipedia
Ok, I love Wikipedia as much as the next guy but the amount of weight that google puts on this site is getting crazy. My search terms that I am going after are "speakers" and "loudspeakers" Can somebody tell me why wikipedia needs the top 8 -10 spots for those terms? is that really a good search result for users of google? More of a rant then a question I know. I just needed to get that off my chest!.
Algorithm Updates | | kevin48030 -
Geolocation: Google only crawls from the US
A question was previously asked about geo-location and specifically if Google crawled from other countries. I could not locate the original question but wanted to share the below information. As of earlier this year Google only crawls from US IP addresses: http://www.youtube.com/watch?v=7paVYBgH0Hw
Algorithm Updates | | RyanKent1