How Is a Page Crawled by Moz When Moz Says 'No Links'?
-
As above, really. I've crawled a new client's site to find the Moz crawler has identified a handful of 404 errors.
The Moz crawler says these pages have '0 linking domains', and OSE has no data for these pages.
So how are these pages being crawled by Moz and what should I advise my client?
-
Thanks again, Peter.
-
That's great, thanks Keri.
-
Hi, no problem. Keri has answered your query re the Moz crawler not finding them.
Peter
-
If you download the CSV report, we list the referring page there. We know it isn't obvious and we're working on making this a little easier to find.
-
Hi Keri,
Thanks for getting back to me (thanks to Peter above, also).
So what is the best way of me finding where these 404 pages are linked from?
I am 99% sure they are internal links that I will ask the client to remove, but would like to avoid manually checking every link on the site if possible.
-
Hi Peter,
I searched for their domain name and about the expected number of results came up.
I searched for these individual URLs that Moz is reporting using the site: command, and they were not returned.
They are also neither in the site's sitemap (HTML page or XML file).
So how is SEOmoz Crawler finding them?
-
We have two completely different crawling systems. We have on crawler for your website, where you enter in your URL and we crawl your site looking for problems. We have a different crawler for OSE that goes and looks at the web in general and crawls for links. The two crawlers do not talk to each other, so it is actually understandable that OSE may not have data on the site. OSE doesn't have the same index size as Google, so we don't see all the links that are actually out there (though we're working all the time on improving this!).
-
Hi, if you put site:xxx.com into Google where xxx.com is your client's domain name, does Google report anything?
Peter
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Internal Links is reporting "0" value
Open Site Explorer is reporting "0" value under "Total Internal Links" as well as "Internal Links" metrics for my website. I have a good interlinking of pages throughout the site.
Link Explorer | | anupjain
Is there an issue with OSE (Open Site Explorer) crawler ? OR There are some crawling problems with our Website's structure ?
Can anybody please give an idea or recommendations on this ?
Domain: https://www.traveltoindia.org/0 -
Link tracking list not finding my links
I am using the Link Tracking List to track new backlinks that I have achieved. However, the report still shows about half of them as not found... even some that have been there for several weeks. I thoroughly tested a couple of them to make sure the links were coded in html, there were no nofollow/noindex rules, the bot was not blocked etc, but still found nothing. What can I do now?
Link Explorer | | Zambezikid0 -
What do these link terms mean?
I'm not sure what all of the different link terms mean: -External, followed links -Internal, nofollowed links -External, nofollowed links -Internal, no followed links -Total linking domains -Followed linking domains What are the differences? Which metric will indicate the total # of other sites that are linking to my site? Thank you!
Link Explorer | | sgarrels0 -
Moz crawling doesn't show all of my pages
Hello, I'm trying to make an SEO on-pages audit of my website. When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ? How can i check and correct this. My website is discoolver.com
Link Explorer | | MK-Discoolver1 -
Open Site Explorer not attributing page authority
Hi there, I have been having an issue with OSE attributing page authority to one of the sites I have been looking at. The homepage page authority is fine but any page after this does not have any according to OSE. I have been in touch with Moz and they have stated it's because there are no external or internal links pointing to any of these pages. However, I know that their are internal links pointing to these pages from the homepage in the nav. The domain is: https://healys.com/ If anyone has any ideas what may be going on that would be great. Thanks!
Link Explorer | | Alice-20160 -
No Internal Links
I have noticed on both OSE and also on the Crawl Test that our site www.advantageservices.net is showing only 1 internal link. I tried looking around as what might cause this but haven't been able to figure it out.
Link Explorer | | Kristine-ASNET0 -
When requesting a CSV for an internal 302 redirect report, the data isn't aligning with the UI. Does anyone have any suggestions?
When requesting a CSV for an internal 302 redirect report, the data isn't aligning with the UI. Does anyone have any suggestions?
Link Explorer | | Merkle-Impaqt0 -
Psst Did you see Open Site Explorer's New Link Building Opportunities Section??
Check out Rand's blog post about it for all the details: http://moz.com/blog/open-site-explorers-new-link-building-opportunities-section. We'd love for you to check it out and leave us your feedback! Thanks,
Link Explorer | | jennita
Jen3