My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at [email protected]! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Even canonical present in my web pages, still moz showing canonical issues.
Even canonical present in my web pages, still moz showing canonical issues.
Moz Local | | prasad.nueve0 -
Validation needed - listing not found on google or facebook, but I'm looking at my google my business listing. Whats going on?
It is telling me "we were unable to validate the following listings with google or facebook", but the information matches my google my business listing exactly (for as much as I can tell) why won't it let me validate? I know I have a google my business listing as I am the manager of it and i synced it to moz local via google my business. So why is it not validating when i know for a fact it is listed in Google? Sharper Impressions Painting Inc 6292 Valleyview Dr, Fishers, IN 46038 (317)423-0230. My other 4 sharper impressions listings were validated (even the kansas city one that is still waiting to be verified on google my business) so I am at a loss as to why this one will not validate. Please advise.
Moz Local | | LiL_C-bus0 -
Facebook page not showing up on mo report
Hi, I've recently set-up a Moz local account and have submitted my business to it. However for some reason my Facebook page is not showing up in the listing report. The Facebook page is active and has all the NAP details but in the listing report, it only gives the option of creating a listing. Am I doing somesting wrong? https://www.facebook.com/surreyroofcare/
Moz Local | | surreyroofcare0 -
What do I do if the business listed in Moz Local Business Listing for Yellow Pages is wrong?
Hey there! I have a business listing set up in Moz Local. The system seems to have accurately found the business on most networks, but on Yellow Pages, it has a completely different business listed (my business is a Pilates studio and it has a soccer club listed). What are the steps I can take to fix this? I can claim my listing on Yellow Pages, but after I do that, how does Moz Local associate the two? Thanks, --eric
Moz Local | | EricOliver0 -
Facebook page has disappeared from moz local
My facebook page is very active and very much complete. This was shown on moz local up until a week ago. I have changed nothing on facebook but now moz local shows that it is 0% complete and only gives the option of creating a listing. How do i rectify?
Moz Local | | Sassandgrace0 -
Insider Pages & City Search Update Not Going Through
Hello, I updated our company phone number about 2 months ago and every aggregator updated except Insider Pages and City Search. They both have empty circles and are showing as Inconsistent when opening up the campaign because they have the old phone number. I thought for awhile it would just take some time, but it doesn't seem to be updating at all. Is it possible that it's stuck? A lot of the others changed to half circles, then full. But this one is just sitting as an empty circle. Thanks, Chris
Moz Local | | Cincinnati_WebTec0 -
You have exceeded your rate limit error while trying to add listing to Mozlocal. What's wrong?
I get the error You have exceeded your rate limit when trying to add my first listing to the MozLocal tool. Can you let me know what's the problem?
Moz Local | | arkenea1 -
Error's while trying to upload listings
Moz Local continues to tell me there are errors with the excel document i'm uploading. So as told I saved it as a .csv and tried to close out and it a notice continues to comes saying " This workbook contains features that will not work or may be removed if you save it in the selected file format. Do you want to continue? " both options don't close the document and I can't close the document without losing the work i've done. What do I do?
Moz Local | | RankPop0