Moz not crawling opencart product pages
-
hi,
i have waited for over 2 weeks now and the crawler only got 8 pages, and is not getting all the open cart pages and products.
any idea of what can be wrong?
im using joomla 2.5.11 and mijoshop 2.0.5 (which uses opencart 1.5.5.1).
thanks
-
Hi Marlon,
There are a lot of different issues that may cause Rogerbot to be unable to crawl your site. For troubleshooting tips, I suggest reading http://moz.com/help/pro/why-can-t-rogerbot-crawl-my-site. Hope that helps!
-
ok thanks
-
Screaming Frog has an option in the menu -
Configuration > Spider > "Ignore Robots.txt"
Tick this box and re-scan and see if you get different results.
And also, may be worth checking this out:
http://moz.com/learn/seo/robotstxtGood luck!
-
hi
thanks for the answer, tried screaming frog now as well and seems it has the same problem,
is it possible to allow only certain spiders/robots to crawl all files, with the robots.txt?
thanks
-
It's hard to say without knowing the domain name -
Have you tried Screaming Frog software to see if it also has issues crawling?
Double check your robots.txt to see if some directories and pages are blocked.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Page Optimization problem in Moz
Hello there, I've been using Moz Pro for a year now, and encountered I've encountered small issues here and there (like some features not working currently, but they start working after a few hours tops) but now I have a bigger issue. Page Optimization feature stopped working and it's dead for five days now. Even my older research is gone. (check the images) Anyone else having this problem, or is it just me? It says that the crawler is blocked or the page isn't working, and when I check the pages in other platforms with crawlers it all goes smooth. All of the pages are working, and all of the crawlers (tested even in Google console) are doing fine. I even had the Moz crawler on 04.07.2019 as the standard week crawl and I got the results. Everything else is working just fine, I have a problem with the Page Optimization feature only. Thanks in advance, Ivan Ga4LKkQ
Moz Bar | | BMGEmployee0 -
If we put the disavow links in google, does MOZ crawl the same links?
I have put bad or spam links in disavow file, but still showing in MOZ backlinks. So, I want to know that Why is MOZ not removing the spam links from their system?
Moz Bar | | insidewebanalytics0 -
Any Moz staff out there with Magento experience?
Would be great to hook up with Moz staff that are Magento-integration savvy. Or maybe qualified users . . . Need to go through Magento Admin in order to correct errors found in Site Crawl. Thanks, Steve
Moz Bar | | spiderx.ing0 -
Crawl-test not doesn't finish
Hello, I have used this crawl-test on 2 website 3 days ago, and it hasn't finished yet. I'm wondering if the crawler is on an infinite loop, or has crashed without sending back an error. I could re-launch the test, but if it's really still crawling, I don't want to loose any work in progress. Is there any way to check the status of a crawl?
Moz Bar | | Nobody16116145880332 -
MOZ Freshweb reported mentions that really does not contain any mention - anyone else experiencing this?
MOZ Freshweb reported links from "traffic.fullcontentrss.com" that doesn't really contain any mention. Is this a MOZ error or is it maybe since fullcontentrss.com has a javascript that pulls data into their own RSS, MOZ ended up crawling partially and as a result not reporting the correct URL? Anyone else experiencing this anomaly?
Moz Bar | | CPR_PTANTONO0 -
URL not returning a page successfully
I am trying to use the on page grader however it is not working with my website. The url is: http://www.britishhardwoods.co.uk/ Can anyone help?
Moz Bar | | British_Hardwoods0 -
Chrome moz toolbar page analysis not loading
Often the chrome moz toolbar page analysis doesn't load just says Please wait until the page finishes loading.
Moz Bar | | genkee0 -
SEOMOZ On page ranking reports update
I've recently made a load of changes to improve the on page ranking report grades for my site. I was givena C grade for this page: http://www.katalysticevents.com/luxury-wedding-teepee-hire-about/ based on two key phrases Wedding Tipi and Wedding Teepeee. I've made all the changes as advised (excluding H1 tag) by the report but when I rereun the report it's picking up old page data and showing the same problems. Is there some kind of cache thing I'm missing here? The confusing thing is I made the same series of changes for this page: www.katalysticevents.com/luxury-marquee-hire-services/ for Luxury marquee and luxury marquee hire and the results are fine...I've got an A grade. Examples are things like ALT image attirbutes which don't seem to show up on the first page but are fine on the second page.
Moz Bar | | EdoubleD
The site is a Word press site and I'm using the "All in one SEO pack" plugin. Can anyone advise?0