My crawl is only crawling only the 1st page. What’s going wrong?
-
When I crawl the website kmtstore.com usiing the moz tool , it crawls the product urls only the fist page and skips the rest of the products on the remainaing pages .
Here are the produclist page urls which has the products on many pages but on crawling only the 1st page products show up . I have crawled this website kmtstore.com yesterday and all the details being given is as per the latest crawl report generated yesterday./
http://kmtstore.com/pl/11099/H2O-Jet-Parts-Flow-Style.aspx
http://kmtstore.com/pl/11103/Pneumatic-Control-Valves.aspx
http://kmtstore.com/pl/11104/Swivel-Joint-Assemblies.aspx
http://kmtstore.com/pl/11105/Cutting-Head-Components/Cutting-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11106/Cutting-Head-Components/Focusing-Tubes.aspx
http://kmtstore.com/pl/11107/Cutting-Head-Components/Cutting-Heads.aspx
http://kmtstore.com/pl/11108/Cutting-Head-Components/Orifice.aspx
http://kmtstore.com/pl/11109/HP-Valves-Fittings-and-Tubing/Tubing.aspx
http://kmtstore.com/pl/11110/HP-Valves-Fittings-and-Tubing/Inline-Water-Filter.aspx
http://kmtstore.com/pl/11111/HP-Valves-Fittings-and-Tubing/High-Pressure-Fittings.aspx
http://kmtstore.com/pl/11112/HP-Valves-Fittings-and-Tubing/Gauges.aspx
http://kmtstore.com/pl/11113/Intensifier-Parts/Electrical.aspx
http://kmtstore.com/pl/11114/Intensifier-Parts/Low-Pressure-Water-Systems.aspx
http://kmtstore.com/pl/11115/Intensifier-Parts/Topworks-Replacement-Parts.aspx
http://kmtstore.com/pl/11116/Intensifier-Parts/Hydraulic-System.aspx
http://kmtstore.com/pl/11117/Intensifier-Parts/Sealing-Head-Replacement-Parts.aspx
http://kmtstore.com/pl/11118/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Coning-and-Threads.aspx
http://kmtstore.com/pl/11119/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Topworks.aspx
http://kmtstore.com/pl/11120/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Lubricants.aspx
http://kmtstore.com/pl/11121/Maintenance-Tools-and-Lubricants/Maintenance-Tools-Pneumatic-Valves.aspx
http://kmtstore.com/pl/11122/Maintenance-Tools-and-Lubricants/Manuals.aspx
http://kmtstore.com/pl/11123/Pneumatic-Control-Valves/Pneumatic-Control-Valve-Replacement-Parts.aspx
http://kmtstore.com/pl/11124/Pneumatic-Control-Valves/Cutting-Head-Nozzle-Tubes.aspx -
Hi there!
Thanks for reaching out! My name is Erin, and I'm on the Moz Help Team. I appologize for any confusion here. I downloaded your crawl CSV for kmtstore.com, and everything looked fine. From what I can tell Google has 296 pages indexed for your site, and we crawled 241.
Our crawler starts with your home page and uses recursive crawling to find pages on your site through the links we find. Thus, your homepage is the only seed we use to crawl your site. We know that Google uses multiple seeds, so it could be possible Google is indexing more pages that way. Since we use recursive crawling, if these product pages aren't linked to your homepage, our crawler won't be able to pick them up.
I hope this helps, and if you have any other questions, please reach out to us at [email protected]! We'll be able to take this conversation offline and answer any other questions for you from there!
Cheers!
Erin
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
3 Locations = 1 Facebook Page or = 3 Facebook Pages
What is the best practice? I have a client with 3 locations with an actual address and not virtual. He has 1 Facebook page with the address and phone number of the main location. Do I add locations in Facebook on that main FB page or do I create 2 more FB pages to have one for each location?
Moz Local | | PSLab0 -
Difference between 'City' and 'Province' on Moz Local business listing page
Sorry, I'm stuck on this one. I'm trying to clean up my business address on Moz Local as I have a duplicated version that ranks better than my verified listing (which is Fulham Nannies). I'm wondering if the problem arises from my entry on Moz Local's business listing for which I've entered 'London' in both the 'City' and 'Location' fields.. I'm not entirely sure that's correct, or what I should in fact enter in the 'province' field other than what I've got at the moment, I.e. London. Should I put something like 'South West London' for province and keep the city entry as London? I'd really appreciate any guidance anyone might have, thanks in advance.
Moz Local | | Moira_Walsh0 -
Facebook page has disappeared from moz local
My facebook page is very active and very much complete. This was shown on moz local up until a week ago. I have changed nothing on facebook but now moz local shows that it is 0% complete and only gives the option of creating a listing. How do i rectify?
Moz Local | | Sassandgrace0 -
How can I add the county (I'm in the UK) to my Facebook page
I am trying to setup a Moz Local campaign but I am unable to change the county (I am in the UK). In the Moz report, my 'state' is listed as 'Cnwll', but it should be 'Cornwall'. My Google + page has the correct information and is verified. My Facebook page has the correct information, apart from the fact that I can't add add a county. Does anyone have any suggestions on where I could go from here? Thanks.
Moz Local | | Nanuq0 -
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
I updated my Google My Business page weeks ago and the name of my business still has not changed in Moz Local. Of course it won't allow me to change the name. Please help.
Moz Local | | NicoleDeLeon0 -
You have exceeded your rate limit error while trying to add listing to Mozlocal. What's wrong?
I get the error You have exceeded your rate limit when trying to add my first listing to the MozLocal tool. Can you let me know what's the problem?
Moz Local | | arkenea1 -
Linking Google Business Page to Moz Local?
I'm working on updating an existing Moz Local account, and one of the listings says that there is "No listing found on Google." However, there is a Google Business page for this listing, it just isn't linked to my account. And the only option Moz gives me is to "create listing" on Google, which takes me to the page to create a Google business listing. So my question is this: how do I link an existing Google Business page to a Moz local listing that is already mostly set up?
Moz Local | | LWSEO0 -
Does the "Unmanaged" number go up immediately after submission?
I just submitted a CSV file with the information about 3 businesses to Moz Local, but I can't tell if it went through properly. The page gave me a message saying "0 are ready to publish. 0 need validation", and didn't list any errors in my listings. However, the number next to "Unmanaged" is still a 0. I know from looking at some of the other questions on here that from the time that Moz sends the information out, it can take a few weeks for the listings to propagate. But I'm not sure whether or not it's supposed to take some amount of time for uploaded listings to show up on here as Unmanaged listings. Should I try resubmitting, or does it just take a few minutes/hours for unmanaged listings to show up?
Moz Local | | BrianAlpert780