URLs are not indexed
-
My website has 0.5 million pages with urls like this- **http://www.mycity4kids.com/Delhi-NCR/collage-painting-classes-%3cnear%3e-shalimar-bagh ****, **none of these urls are indexed.
Question 1- What can be the possible reason for this issue? Users see this url as : http://www.mycity4kids.com/Delhi-NCR/collage-painting-classes-<near>-shalimar-bagh</near>
The symbol "<" and ">" get converted into "%3c" and "%3e" respectively, is this the reason for these urls not getting indexed? -
Hi Prashant,
If these URLs are only created when a user searches for a term, but are not linked to anywhere in HTML either on your site or elsewhere on the Internet, this is a very good reason why they are not indexed.
Google does not usually perform queries on a site (e.g. fill in forms) to "discover" what content might be displayed when those forms are filled in. It's tried and tested method of crawling is just that - crawling links and text in HTML. It has become more adventurous with different technology and sometimes finds things that it wouldn't have previously, but linking is still the primary way to ensure something gets crawled.
In many cases, you wouldn't want Google finding content that it has to perform queries / fill in forms or searches to get to: this is how some sites create massive amounts of duplicate content by accident. So in a way, Google is doing everyone a favour by not indexing URLs like this.
We have submitted these urls on google webmaster using a "sitemap" for indexing still none of them are indexed.
I tend to think of sitemaps like road maps: they're a guide. The site itself is the road. If a map tells me that I can drive across a river but when I get there, this is no bridge, I'm not going to drive across the river. Maybe I will if I have a huge four-wheel-drive car with a snorkel, but possibly not. Maybe Google will index URLs it can't find on the web itself, but possibly not.
If you want the URLs to be found, link to them
Cheers,
Jane
-
Just Google the URL it can't be simpler, if there is a result it's indexed.
-
Thanks Martijn
Could you please tell how many ways are there to check whether a page is indexed or not?
Someone on a different forum told me that we can check "indexed status" of a url by directly googling that url, there is no need to use "site:" operator.
-
Hi Prashant,
Then that's definitely your issue, I wouldn't index any pages to be honest that I couldn't find in any navigation besides self finding the pages. I would link from any related pages to the pages that you want to have indexed.
-
Thanks Martijn,
Yes, if a user enter a search term then these URL get formed. for example if you visit : www.mycity4kids.com and enter Collage Painting Classes in 1st search box and Shalimar Bagh in 2nd search box then you will see this:** http://www.mycity4kids.com/Delhi-NCR/collage-painting-classes-<near>-shalimar-bagh</near>** url in address bar.
We have submitted these urls on google webmaster using a "sitemap" for indexing still none of them are indexed.
-
Hi Prashant,
Can these pages be found via the navigation or do you have to enter a search term to get to them? I can't find any links to the first page you've listed in your question. If that's the case than it's quite normal that Google didn't index the page as they just aren't able to find the page in the first place.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Index, follow on a paginated page with a different rel=canonical URL
Hello, I have a question about meta robots ="index, follow" and rel=canonical on category page pagination. Should the sorted page be <meta name="robots" content="index,follow"></meta name="robots" content="index,follow"> since the rel="canonical" is pointing to a separate page that is different from the URL? Any thoughts on this topic would be awesome. Thanks. Main Category Page
Intermediate & Advanced SEO | | Choice
https://www.site.com/category/
<meta name="robots" content="index,follow"><link rel="canonical" href="https: www.site.com="" category="" "=""></link rel="canonical" href="https:></meta name="robots" content="index,follow"> Sorted Page
https://www.site.com/category/?p=2&dir=asc&order=name
<meta name="robots" content="index, follow"=""><link rel="canonical" href="https: www.site.com="" category="" ?p="2""></link rel="canonical" href="https:></meta name="robots" content="index,> As you can see, the meta robots is telling Google to index https://www.site.com/category/?p=2&dir=asc&order=name , yet saying the canonical page is https://www.site.com/category/?p=2 .0 -
How to get a large number of urls out of Google's Index when there are no pages to noindex tag?
Hi, I'm working with a site that has created a large group of urls (150,000) that have crept into Google's index. If these urls actually existed as pages, which they don't, I'd just noindex tag them and over time the number would drift down. The thing is, they created them through a complicated internal linking arrangement that adds affiliate code to the links and forwards them to the affiliate. GoogleBot would crawl a link that looks like it's to the client's same domain and wind up on Amazon or somewhere else with some affiiiate code. GoogleBot would then grab the original link on the clients domain and index it... even though the page served is on Amazon or somewhere else. Ergo, I don't have a page to noindex tag. I have to get this 150K block of cruft out of Google's index, but without actual pages to noindex tag, it's a bit of a puzzler. Any ideas? Thanks! Best... Michael P.S., All 150K urls seem to share the same url pattern... exmpledomain.com/item/... so /item/ is common to all of them, if that helps.
Intermediate & Advanced SEO | | 945010 -
Competing URLs
Hi We have a number of blogs that compete with our homepage for some keywords/phrases. The URLs of the blogs contain the keywords/phrases. I would like to re-work the blogs so that they target different keywords that don't compete and are more relevant. Should I change the URLs as I think this is what is mainly causing the issue? If so, should I 301 old URL's to the homepage? For example, say we we're a site that specialised in selling plastic cups. Currently there is a blog with the URL www.mysite.com/plastic-cups that outranks the homepage for _plastic cups. _The blog isn't particularly relevant to plastic cups and the homepage should rank for this term. How should I let Google know that it is the homepage that is most relevant for this term? Thanks
Intermediate & Advanced SEO | | Buffalo_71 -
I'm noticing that URL that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before?
I'm noticing that URLs that were once indexed by Google are suddenly getting dropped without any error messages in Webmasters Tools, has anyone seen issues like this before? Here's an example:
Intermediate & Advanced SEO | | nystromandy
http://www.thefader.com/2017/01/11/the-carter-documentary-lil-wayne-black-lives-matter0 -
Google not indexing images
Hi there, We have a strange issue at a client website (www.rubbermagazijn.nl). Webpage are indexed by Google but images are not, and have never been since the site went live in '12 (We recently started SEO work on this client). Similar sites like www.damenrubber.nl are being indexed correctly. We have correct robots and sitemap setup and directions. Fetch as google (Search Console) shows all images displayed correctly (despite scripted mouseover on the page) Client doesn't use CDN Search console shows 2k images indexed (out of 18k+) but a site:rubbermagazijn.nl query shows a couple of images from PDF files and some of the thumbnails, but no productimages or category images from homepage. (product page example: http://www.rubbermagazijn.nl/collectie/slangen/olie-benzineslangen/7703_zwart_nbr-oliebestendig-6mm-l-1000mm.html) We've changed the filenames from non-descriptive names to descriptive names, without any result. Descriptive alt texts were added We're at a loss. Has anyone encountered a similar issue before, and do you have any advice? I'd be happy to provide more information if needed. CBqqw
Intermediate & Advanced SEO | | Adriaan.Multiply0 -
URL rewrites
We have a problem whereby a number of our urls are adressable from different urls - I'm told because of a quirk of developing in .net. e.g. mysite/FundComparison mysite/Fund-comparison mysite/fund-comparison We asked our supplier who hosts this section of our site to do some url rewrites so that the duplicates would 301 to the correct url. They're on IIS 6.0 and are not ready to upgrade to IIS 7.0 (my recommendation, which makes it easier for them to do the rewrite using the rewrite module). They said it would take 6-8 weeks to implement a web controller to do this. "The bulk of the time for this implementation is in the build of the engine + the addition of all the possible permutations of the URL to redirect to the proper URL." This sounds absolutely insane to me. I would have thought it could be done in a matter of hours. What do people think?
Intermediate & Advanced SEO | | SearchPM0 -
Sitemap not indexing pages
My website has about 5000 pages submitted in the sitemap but only 900 being indexed. When I checked Google Webmaster Tools about a week ago 4500 pages were being indexed. Any suggestions about what happened or how to fix it? Thanks!
Intermediate & Advanced SEO | | theLotter0 -
2 Language Versions on Same URL
A site we are working on is a large gift retailer in Canada. They have a language option for French, but the page URLs are the same. If you click 'French' in the header, a cookie is set and then all pages are dynamically served the French content (and all nav/site elements of course change to French). The URLs then are exactly the same as it's the cookie that determines the language option to serve. e.g. www.site.ca/index.php?category=7&product=99.... would be the same regardless of if I'm set for English or French. Question: Does this setup have a negative impact on any SEO factors? The site has several thousand pages.
Intermediate & Advanced SEO | | BMGSEO0