Sudden increase in number of indexed URLs. How ca I know what URLs these are?
-
We saw a spike in the total number of indexed URLs (17,000 to 165,000)--what would be the most efficient way to find out what the newly indexed URLs are?
-
You can also try searching your URL in google using exact paramaters...
inurl:http://www.yourdomain.com
or if you dont have a www.
inurl:http://yourdomain.com
-
The rest are most likely put into the supplemental index (are duplicates). I'd review GWT and analyze crawl stats, crawl errors, and html improvements.
Do you have 165k pages on your site? If not, it's probably some sort of error.
-
Hi Oleg,
Just tried that, but, it is only showing 300 URLs for the past week and 600 for the past month..
-
You can use the SEOmoz OSE you can use your google webmaster tools and you can also use Majestic . 17,000 to 165,000 indexed pages is a huge jump.
-
Search site:yourdomain.com, click on "Search Tools" in the left column and choose "Last week" (or within whatever period you saw the increase in indexed pages).
Or, you can choose "Past Year" and then choose "Sort by Date" - this will give you the latest links indexed in order.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Number of topics
Hello, is the number of topics you cover important or is it best to cover only a few topics but in more depth ? Thank you,
Intermediate & Advanced SEO | | seoanalytics0 -
Changing URL to a subdomain?
Hi there, I had a website www.footballshirtcollective.com that has been live since July. It contains both content and eCommerce. I am now separating out the content so that; 1. The master domain is www.footballshirtcollective.com (content) pointing to a new site 2. Subdomain is store.footballshirtcollective.com (ecommerce) - pointing to the existing site. What do you advise I can do to minimise the impact on my search? Many thanks Mike
Intermediate & Advanced SEO | | mjmaxwell0 -
How to de-index old URLs after redesigning the website?
Thank you for reading. After redesigning my website (5 months ago) in my crawl reports (Moz, Search Console) I still get tons of 404 pages which all seems to be the URLs from my previous website (same root domain). It would be nonsense to 301 redirect them as there are to many URLs. (or would it be nonsense?) What is the best way to deal with this issue?
Intermediate & Advanced SEO | | Chemometec0 -
Website Suddenly dropping rank
Morning Moz Fans: My URL is: http://goo.gl/Dhbjwj According to MOZ, which we are tracking this URL with, somewhere between the 3rd Feb and 10th the domain went from being fairly well indexed to being dropped to pages further back than 6-7, for pretty much everything, even the company name was only registering at the bottom of page one. Around this time we were transferring the website from .php into wordpress, so we were creating new pages, called by the same names and all the same content but we created the wordpress area in a sub domain of the website. Again around this time we had an issue with the blog area and had to take it down for 4-5 weeks due to some errors which meant google wouldn't have been able to crawl these pages properly, but the rest of the website was up and running. We also discovered recently that the company have and use this domain http://goo.gl/5JvDUH So my question is, what do you think caused the problem? has it been premaritally penalised? is there a way I can get google to specifically look at it and is there any more i can do?Thank you for your help
Intermediate & Advanced SEO | | popcreativeltd0 -
Removing Parameterized URLs from Google Index
We have duplicate eCommerce websites, and we are in the process of implementing cross-domain canonicals. (We can't 301 - both sites are major brands). So far, this is working well - rankings are improving dramatically in most cases. However, what we are seeing in some cases is that Google has indexed a parameterized page for the site being canonicaled (this is the site that is getting the canonical tag - the "from" page). When this happens, both sites are being ranked, and the parameterized page appears to be blocking the canonical. The question is, how do I remove canonicaled pages from Google's index? If Google doesn't crawl the page in question, it never sees the canonical tag, and we still have duplicate content. Example: A. www.domain2.com/productname.cfm%3FclickSource%3DXSELL_PR is ranked at #35, and B. www.domain1.com/productname.cfm is ranked at #12. (yes, I know that upper case is bad. We fixed that too.) Page A has the canonical tag, but page B's rank didn't improve. I know that there are no guarantees that it will improve, but I am seeing a pattern. Page A appears to be preventing Google from passing link juice via canonical. If Google doesn't crawl Page A, it can't see the rel=canonical tag. We likely have thousands of pages like this. Any ideas? Does it make sense to block the "clicksource" parameter in GWT? That kind of scares me.
Intermediate & Advanced SEO | | AMHC0 -
What may cause a page not to be indexed (be de-indexed)?
Hi All, I have a main category page, a landing page, that does not appear in the SERPS at all (even if I serach for a whole sentence from it). This page once ranked high. What may cause such a punishment for a specific page? Thanks
Intermediate & Advanced SEO | | BeytzNet0 -
Indexed non existent pages, problem appeared after we 301d the url/index to the url.
I recently read that if a site has 2 pages that are live such as: http://www.url.com/index and http://www.url.com/ will come up as duplicate if they are both live... I read that it's best to 301 redirect the http://www.url.com/index and http://www.url.com/. I read that this helps avoid duplicate content and keep all the link juice on one page. We did the 301 for one of our clients and we got about 20,000 errors that did not exist. The errors are of pages that are indexed but do not exist on the server. We are assuming that these indexed (nonexistent) pages are somehow linked to the http://www.url.com/index The links are showing 200 OK. We took off the 301 redirect from the http://www.url.com/index page however now we still have 2 exaact pages, www.url.com/index and http://www.url.com/. What is the best way to solve this issue?
Intermediate & Advanced SEO | | Bryan_Loconto0 -
Changing Site URLs
I am working on a new client that hasn't implemented any SEO previously. The site has terrible url nomenclature and I am wondering if it is worth it to try and change it. Will I lose rankings? What is the best url naming structure? Here's the website http://www.formica.com/en/home/TradeLanding.aspx. (I am only working on the North America site.) Thanks!
Intermediate & Advanced SEO | | AlightAnalytics0