"Our crawler was not able to access the robots.txt file on your site."
-
Hi Mozzers!
I've received an error message saying the site can't be crawled because Moz is unable to access the robots.txt. I've spoken to the webmaster and he can't understand why the robot.txt can't be accessed as this seems to be fine:
https://k3syspro.com/robots.txt
and Google isn't flagging anything up to us.
Does anyone know why this may be?
Thanks,
Matthew
-
Hey K3,
Looks like you have bigger fish to fry, your host domain is throwing a 503 server error. No server = no content = no robots.txt will be able to be found until that is resolved. Error still happening as of this post: http://downforeveryoneorjustme.com/https://k3syspro.com
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to remove staging Urls from being shown in MOZ as critical crawler issues?
Hi MOZs, A few months ago we were updating our website and used a staging site to do so. Soon after we realized that it was indexed by G and shown in Search so we removed it to be indexed. The staging site has since not been visible in search except for 2 links that I just found out about today. My question is how to remove the staging site URLs being shown in MOZ as crawl issues or remove completely? I want to do this because the digital reports are connected to MOZ and it shows there are currently more than 400x 4xx errors which are all because of the staging URLs. I already marked to ignore the issues but they are still showing in. Any help would be much appreciated! Thanks
Product Support | | Strausberg0 -
Merging two Sites into One - how is Final Domain Authority Calculated?
Hi, If we merge two sites say, one with DA 35 and another 20 what will be resulting DA of the merged site? Secondly, will the result vary based on which Domain is kept and which one is 301ed? ps :- all content is transferred from one to another using 301. Regards,
Product Support | | Praveenshah10 -
Rogerbot not crawling our site
Has anyone else had issues with Roger crawling your site in the last few weeks? It shows only 2 pages crawled. I was able to crawl the site using Screaming Frog with no problem and we are not specifically blocking Roger via robots.txt or any other method. Has anyone encountered this issue? Any suggestions?
Product Support | | cckapow0 -
Giving others like employees admin access ?
Hi I have my master logins for Moz but just wandering if i can grant a degree of access to employees via their own logins (just 1 person at this stage) since i dont want to share mine (since use for other stuff too) and ideally dont want them to have full admin permission (such as being able to delete campaigns) ? Cheers Dan
Product Support | | Dan-Lawrence0 -
My campaigns are all missing and can't access archives
"Oops! Try refreshing the page, if that doesn't work, please click here to contact our help team." Yu0PY7g
Product Support | | matthewjharris0 -
"Our connection to your Google Analytics account was lost" keeps occurring
We've reauthorized our Google Analytics accounts, but we still keep getting the message "Our connection to your Google Analytics account was lost. Please reauthorize now." This is happening to a handful of our campaigns, not all. Has anyone else encountered this or know what is going on? It says I won't lose any data, but why does this keep happening?
Product Support | | ArtUnlimited0 -
Is it possible to split a moz account. We have 5 sites and one has been sold to another company. Can I split that site onto its own new account?
Is it possible to split a moz account. We have 5 sites and one has been sold to another company. Can I split that site onto its own new account?
Product Support | | BuyandSell0 -
How to force MOZ crawlers to search in the right area for local SEO
We do local SEO over here and so our keywords are always followed by a city name, like so: "auto repair+Springfield". If there's ever any confusion within SERPS which "Springfield" the searcher is looking for we can depend on Google to pull query location to assume it's the nearest to them (it does, and is usually correct). And furthermore, if the office ever needs to check for rankings by hand (we're not in Springfield), we just set the query location to "Springfield" and our results match their. However, MOZ doesn't set a query location. Which makes sense, I suppose, because their crawlers are up in the Bay. What I'm saying is that oftentimes there's a discrepancy between how MOZ tells me they rank, and how I see them ranking, the business owner sees them ranking, and how the customer finds them ranking. As such, we can't send them reports through MOZ 😞 Has anyone else had this problem and found a solution? Should the keyword be followed by a zip code instead of a city name? This seems like it would solve the location query problem but it may not provide the same results as service+city name.
Product Support | | cwtaylor0