Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
-
Client site is 15 + pages. New site had noindex/nofollow removed prior to last two crawls.
-
Okay I just did a test crawl with 'Screaming Frog' SEO spider - no issues
Robots.txt - not blocking it.
Headers - not blocking them.
Therefore definitely look at Keri's post as it's something SEOmoz's end, so check out that thread for some possible answers
-
Sure. The site is bayareadrainage.com . Thanks in advance.
-
If GA means Google Analytics, Google Analytics doesn't crawl pages -- it reports visits to pages that have the GA code on them. You can have the content behind a password and blocked in robots.txt and if GA code is there, it would still track people.
https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages lists some common reasons why only one page is crawled. If you still have problems with crawling, send a note to [email protected] and we'll help you figure things out.
-
Are you able to post the URL? That way I, or another Mozzer could check the crawlability and just check you're not accidentally blocking robots another way?
I would also check your robots.txt to make sure you're not Disallowing all your site.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
"Noindex, follow" for thin pages?
Hey there Mozzers, I have a question regarding Thin pages. Unfortunately, we have Thin pages, almost empty to be honest. I have the idea to ask the dev team to do "noindex, follow" on these pages. What do you think? Has someone faced this situation before? Will appreciate your input!
Technical SEO | | Europarl_SEO_Team0 -
MOving from one page format to another
I have a page that I optimized for a valuable keyword (at least it is to me!) and it gets very good traffic. Problem is that the page needs to constantly updated with new "news" related to the topic area. I have developed a page-type that will have the same static content as the original page, but it will now automtaically index related content from the rest of the site so the "news" section stay fresh and up-to-date. I am now okay to do something like permanently redirect that page to the newly built one? I have also run the newly designed page through tests on Moz and it receives on receives an A-grade from the on-page optimization tool. I am just wanting to ensure I cover all bases, because I do not want to risk losing the top search result and all that valuable traffic.
Technical SEO | | kevgrand0 -
Site Migration between CMS's
Hi There, I have a technical question about migrating CMS's but not servers. My client has site A on Joomla install, He want's ot migrate to Wordpress and we will call this site B. As he has a lot of old content on site A he doesn't want to lose, he has put site B (wordpress install) on a subdirectory site.com/siteb (for example). and will use a htaccess to forward the root domain to this wordpress site. Therefore anyone going to www.site.com will see the new wordpress site and the old content and joomla install will sit on the root of the server. Will Google have an issue with this? Will it even find the old content? what are the issues for the new site and new content? Look forward getting your guys input
Technical SEO | | nezona1 -
Why can no tool crawl this site?
I am trying to perform a crawl analysis on a client's website at https://www.bravosolution.com I have tried to crawl it with IIS for SEO, Sreaming Frog and Xenu and not one of them makes it further than the home page of the site. There is nothing I can see in the robots.txt that is blocking these agents. As far as I can see, Google is able to crawl the site although they have noticed a significant drop in organic traffic. Any advise would be very welcome Regards Danny
Technical SEO | | richdan0 -
Crawl Diagnostics and Duplicate Page Title
SOMOZ crawl our web site and say we have no duplicate page title but Google Webmaster Tool says we have 641 duplicate page titles, Which one is right?
Technical SEO | | iskq0 -
NOINDEX,FOLLOW on product pages
Hi Can I have people's thoughts on something please. We sell wedding stationery and whilst we can generate lots of good content describing a particular range of stationery we can't relistically differentiate at a product level. So imagine we have three ranges Range 1 - A Bird Range 2 - A Heart Range 3 - A Flower Within each of these ranges we would have invitations, menus, place cards, magnets etc. The ranges vary quite alot so we can write good textual keyword rich descriptions that attract traffic (i.e. one about the bird, one about the heart and one about the flower). However the individual products within a range just reflect the design for the range as a whole (as all items in a range match). Therefore we can't just copy the content down to the product level and if we just describe the generic attributes of the products they will alll be very similar. We have over 1,000 "products" easily so I am conscious of creating too much duplication over the site in case Mr Panda comes to call. So I was thinking that I "might" NOINDEX, FOLLOW the product pages to avoid this duplication and put lots of effort into making my category pages much better and content rich. The site would be smaller in the index BUT I do not really expect to generate traffic from the product pages because they are not branded items and any searches looking for particular features of our stationery would be picked up, much more effectively, by the category pages. Any thoughts on this one? Gary
Technical SEO | | gtrotter6660 -
Are there any SEO implications if a page does two 301s and then a 304?
Curious to see if this is a positive or negative thing for SEO...or even perhaps, neutral. h9SZz
Technical SEO | | RodrigoStockebrand0 -
How can I get Google to crawl my site daily?
I was wndering if there was a trick to getting google to crawl my website daily?
Technical SEO | | labradoodlelocator0