Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
-
Client site is 15 + pages. New site had noindex/nofollow removed prior to last two crawls.
-
Okay I just did a test crawl with 'Screaming Frog' SEO spider - no issues
Robots.txt - not blocking it.
Headers - not blocking them.
Therefore definitely look at Keri's post as it's something SEOmoz's end, so check out that thread for some possible answers
-
Sure. The site is bayareadrainage.com . Thanks in advance.
-
If GA means Google Analytics, Google Analytics doesn't crawl pages -- it reports visits to pages that have the GA code on them. You can have the content behind a password and blocked in robots.txt and if GA code is there, it would still track people.
https://seomoz.zendesk.com/entries/409821-why-isn-t-my-site-being-crawled-you-re-not-crawling-all-my-pages lists some common reasons why only one page is crawled. If you still have problems with crawling, send a note to [email protected] and we'll help you figure things out.
-
Are you able to post the URL? That way I, or another Mozzer could check the crawlability and just check you're not accidentally blocking robots another way?
I would also check your robots.txt to make sure you're not Disallowing all your site.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Site not getting indexed by googlebot.
The following question is in regards to http://footeschool.org/. This site is not getting indexed with google(googlebot) This only happens when the user agent is set googlebot. This is a recent issue. We are using DNN as CMS. Are there any suggestion to help resolve this issue?
Technical SEO | | bcmull0 -
Should I Use Two Domains for Multi Language Sites?
I have an immigration attorney that wants a website in English and another in Spanish. We're going to have some of the website content from the English site translated via a translator to make it true, conversation spanish (automatic translators are good, but not perfect, and we want perfect). So my question is do you think we should use two different domains (englishsite.com, spanishsite.com), a subdomain (spanishsite.englishsite.com) or maybe just a separate section of the regular site (englishsite.com/spanishcontent)? My thought would be either a subdomain or a separate section so that we're not splitting PR.
Technical SEO | | atstickel120 -
Will sitemap generated in Yoast for a combined wordpress/magento site map entire site ?
Hi For an ecommerce site thats been developed via a combination of wordpress and magento and has yoast installed, will the sitemap (& other yoast features) map (& apply to) the entire site or just wordpress aspects ? In other words does one need to do anything else to have a full sitemap for a combined magento/wordpress site or will Yoast cover it all ? This link seems to suggest should be fine but seeing if anyone else encountered this and had problems or if straightforward ? http://fishpig.co.uk/wordpress-integration/docs/plugins.html cheers dan
Technical SEO | | Dan-Lawrence0 -
Using Rel Nofollow on Duplicate Pages
Hi there, I have a rather large site that has duplicate content on many pages due to how it's being spidered by google. I was hoping I could set the internal link to this page as "nofollow." My question is that I have hundreds of other sites with backlinks to these duplicate content pages.. will this affect me negatively if I tell google not to index the duplicated pages?
Technical SEO | | trialminecraftserverfinder0 -
H1 - site name or page title?
Hi all, I have always used h1 tags for the site name and then h2 tags for the page title, thus: Bob's Chunky Bacon Store Smoked Bacon Bacon bacon bacon bacon etc... My reasoning for this is I believe it is semantically correct. The h1 represents a book's title and the h2 the name of a chapter. I seem to have read, in a few places, that my h1 should be the page title: Bob's Chunky Bacon Store Smoked Bacon Bacon bacon bacon bacon etc... From a SEO view, which of these is the better approach? Thanks in advance for any input.
Technical SEO | | jimneath0 -
Photography Sites with Same Developer - Why Is One Ranking & Other Not?
I'm currently confused about the difference in ranking between two competing sites, created by the same agency. http://jmayphoto.com/index2.php#!/home (302 redirected from http://jmayphoto.com...yeah) is not ranking well, and I'm not surprised. However, competitor http://www.shanrenee.com/ is ranking within the top 5 spots for a primary target keyword (dallas wedding photographer) and I don't understand how it's doing so well. I definitely see differences, but not enough to explain how Shan Renee is one page. What am I missing?
Technical SEO | | BrittanyHighland0 -
Getting on to page 1 again
I have recently set up a website, and as soon as it got indexed by google, I ranked on the first page at number 10... However, Ever since I have started trying to backlink.. the more I seem to do, the more I seem to drop down the rankings! The same for both Bing and Yahoo... I am really not sure what the problem is. My site is www.arilinegamesatc.com, and the ranking words are airline games and airline game. Any help woudl be greatly appreciated!
Technical SEO | | rolls1230 -
Crawl Errors for duplicate titles/content when canonicalised or noindexed
Hi there, I run an ecommerce store and we've recently started changing the way we handle pagination links and canonical links. We run Magento, so each category eg /shoes has a number of parameters and pages depending on the number of products in the category. For example /shoes?mode=grid will display products in grid view, /shoes?mode=grid&p=2 is page 2 in grid mode. Previously, all URL variations per category were canonicalised to /shoes. Now, we've been advised to paginate the base URLs with page number only. So /shoes has a pagination next link to /shoes?p=2, page 2 has a prev link to /shoes and a next link to /shoes?p=3. When any other parameter is introduced (such as mode=grid) we canonicalise that back to the main category URL of /shoes and put a noindex meta tag on the page. However, SEOMoz is picking up duplicate title warnings for urls like /shoes?p=2 and /shoes?mode=grid&p=2 despite the latter being canonicalised and having a noindex tag. Presumably search engines will look at the canonical and the noindex tag so this shouldn't be an issue. Is that correct, or should I be concerned by these errors? Thanks.
Technical SEO | | Fergus_Macdonald0