How do i stop Moz from indexing my dev site?
-
Hey,
I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.
How do i stop MOZ including it in its reports?
Cheers everyone.
Bart -
Hi,
To stop moz crawler you have to mention below text in robots.txt for dev.domain.com
User-Agent: rogerbot Disallow: /
Hope this helps.
Thanks
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I’d like to set up a Moz campaign that crawls just the primary website, not subdomains
Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.
Getting Started | | blueprintatl0 -
Where can I find Moz Rank for my websites?
I am unable to find an option in Moz Pro where I can find Moz rank for my website. Please help me to find that.
Getting Started | | TapanSant1 -
Anyone use On-Page Grades in Moz? Please Give us Feedback!
Jackie from Moz here. We're looking to get some feedback on the letter grades we use for On-Page Grades (in the application and in the Research Tools). We made this super easy survey to collect our Community's thoughts. Tell us what you think! Thanks for all your help!
Getting Started | | JackieRae2 -
What Moz tool is best to find reasons google has not spidered by site
I just joined Moz and am trying to use the tools however, when I attempt to do so every link comes to a that only allows me access to post questions here. If anyone can tell me what tool is best to find reasons google has not indexed my site, I would greatly appreciate the help. Also if anyone knows why I am keep getting routed to this forum when I try to use any of the tools, I would also appreciate help with this. So far Moz is very frustrating.
Getting Started | | Johndeeray19640 -
Moz not reading Visit to my website
Hello , i have conencted moz to My new ecommerce website , MOZ is not reading any visits , although i can see the visit on google analytics and i have moz connected to My analytics as well, why i cant see the visits ? need your help please 🙂
Getting Started | | moeayyad0 -
Google + Business Page and Moz Pro
Is there a way to connect my Google + Business page with Moz? The only option I have available to connect when I press the button is my personal Google + Page. Thanks!
Getting Started | | BenhardtConstruction0 -
How do get Moz to spider a Development site PRE LAUNCH?
Hi, Does anyone know how we could get Moz to browse a development site before launch? But without Google and other engines indexing it? Thanks
Getting Started | | bjs20100 -
Link Detox or I can use Open Site Explorer for tracking down bad links?
Here's the thing. I need to find bad external links pointing to my site. Is Link Detox the only option or I can actually use Open Site Explorer for that. If OSE is an option, please give me an idea how I need to go about it. Thanks.
Getting Started | | VinceWicks0