Why xml generator is not detecting all my urls?
-
Hi Mozzers,
After adding 3 new pages to example.com, when generating the xml sitemap, Iwasn't able to locate those 3 new url. This is the first time it is happening. I have checked the meta tags of these pages and they are fine. No meta robots setup!
Any thoughts or idea why this is happening? how to fix this?
Thanks!
-
Thank you Guys!
I resolved the issue! I think it was a caching issue!
-
Like Matt says, could it be that there is a robots.txt file that could block these URLs? Could it be that your XML sitemap generator is using cached results for your site. Can the pages be found on the page via links on other pages?
-
Are they linked to a page that the crawler can find?
What are you generating it with? Online tools like xml-sitemaps.com act like a spider - if they can't crawl from the homepage to the new page, they simply won't find it.
Internal tools, like a Wordpress plugin, should actually find those orphan pages.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
What to do with existing URL when replatforming and new URL is the same?
We are changing CMS from WordPress to Uberflip. If there is a URL that remains the same I believe we should not create a redirect. However, what happens to the old page? Should it be deleted?
Technical SEO | | maland0 -
How to add 301 for many urls
Hi I need to redirect many urls in a website and I was wondering if instead of doing it one by one there is a way to get it the other way round.... Redirect all pages but a few. I get a feeling this is not possible, but prefer asking just in case. Thanks for any feedback
Technical SEO | | turismodevino10 -
Dynamic vs. static URLs
Hello Everyone, I'm new here on MOZ and just getting back into SEO (a little bit) after not doing anything 'myself' for a couple of years. Currently my individual URLs show as: https://www.example.com/index.php?l=product_detail&p=107 (dynamic responsive site). I can switch it to a static site, so the individual product pages read as:https://www.example.com/catalog/category name/product name-107.html It's still a long URL, but it would be keyword rich. Some of my current dynamic pages are indexed,and due to an upgrade I had to do several months back, I already have some redirects (301) from my php extensions to the one listed above. This is my long explanation to my following questions: Does having a dynamic or static site matter when ranking in search engines I already have some redirects coming my older site to this dynamic site, so I would have to make more directs from the dynamic site to my static site - is this okay to do? I'm really at a loss, a couple of years ago, I ranked 1-3 (on Page 1) on Google for all my keywords, (all White Hat work), and now I'm into great abyss of no mans land of the internet (ranked on Page 3+) Thank you for any and all help from everyone! ~Sandra
Technical SEO | | rankmenow0 -
Redirect URLS with 301 twice
Hello, I had asked my client to ask her web developer to move to a more simplified URL structure. There was a folder called "home" after the root which served no purpose. I asked for the URLs to be redirected using 301 to the new URLs which did not have this structure. However, the web developer didn't agree and decided to just rename the "home" folder "p". I don't know why he did this. We argued the case and he then created the URL structure we wanted. Initially he had 301 redirected the old URLS (the one with "Home") to his new version (the one with the "p"). When we asked for the more simplified URL after arguing, he just redirected all the "p" URLS to the PAGE NOT FOUND. However, remember, all the original URLs are now being redirected to the PAGE NOT FOUND as a result. The problems I see are these unless he redirects again: The new simplified URLS have to start from scratch to rank 2)We have duplicated content - two URLs with the same content Customers clicking products in the SERPs will currently find that they are being redirect to the 404 page. I understand that redirection has to occur but my questions are these: Is it ok to redirect twice with 301 - so old URL to the "p" version then to final simplified version. Will link juice be lost doing this twice? If he redirects from the original URLS to the final version missing out the "p" version, what should happen to the "p" version - they are currently indexed. Any help would be appreciated. Thanks
Technical SEO | | AL123al0 -
Changing URL - Ranking Disappeared?
Hi All, I named a page URL /plectrums/ within the back end framework. But then decided to change it to /personalised-plectrums/ I resubmitted a GWT sitemap and 301 redirected plectrums -> personalised-plectrums My ranking for personalised plectrums has disappeared and has not come back does anyone know why this is? Or is there something I have missed? Lewis
Technical SEO | | SO_UK0 -
Rogue url foung in webmaster toos
Buon Giorno from 2 degrees C thick fog wetherby UK 🙂 On this site www.davidclick.com I ran a crawl test and came across a url that doesnt exist in my site, the findings are illustrated here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/rogue-urlcopy_zps6c58ee46.jpg The plot thickens... the source of the referring traffic to a page that doesnt exist can be seen here: http://i216.photobucket.com/albums/cc53/zymurgy_bucket/rogue-link-source_zpsc70a34fc.jpg My intial thoughts rae to disavow via this tool:
Technical SEO | | Nightwing
https://www.google.com/webmasters/tools/disavow-links-main So my question is please: Is this sinister or should I just sit back drink a cup of horlicks and return to a Zen like status of inner peace? Any insights welcome 😉 Grazie tanto, David0 -
Duplicate content error - same URL
Hi, One of my sites is reporting a duplicate content and page title error. But it is the same page? And the home page at that. The only difference in the error report is a trailing slash. www.{mysite}.co.uk www.{mysite}.co.uk/ Is this an easy htaccess fix? Many thanks TT
Technical SEO | | TheTub1 -
How to proceed with domain switch AND url change
Hi, in a few weeks we'll do a major change on our website. This involves over 1.5 million pages indexed in Google driving substantial amount of our traffic. Basically we have 2 types of changes: subdomain switches to domain:
Technical SEO | | TruvoDirectories
ex. product.company.com will become www.product.com
for this we know how to manage DNS and Apache rules different url patterns, basically replacing ugly urls by pretty urls
for this we have advanced 301-mapping rules set up Here is the question - what is best way to proceed with these 2 changes in order to preserve rankings and organic traffic: Do both changes simultaneously? First do url changes, than the domain switch Can you please share your thoughts?0