Using Site Maps Correctly
-
Hello
I'm looking to submit a sitemap for a post driven site with over 5000 pages.
The site hasn't got a sitemap but it is indexed by google - will submitting a sitemap make a difference at this stage?
Also, most free sitemap tools only go up to 5000 pages, and I'm thinking I would try a sitemap using a free version of the tool before I buy one - If my site is 5500 pages but I only submit a sitemap for 5000 (I have no control of which pages get included in the sitemap) would this have a negative effect for the pages that didn't get included?
Thanks
-
Submitting a sitemap in Webmaster Console is always a good idea at any stage. If your website URLs are crawled and indexed in search engines than there will be no negative impact of it but in the longer run if you add more pages sitemap will defiantly a help.
If you are using CMS like WordPress, Joomla, Zencart or any other they all have extensions and plugins in their directory that will help you generate the sitemap of your current site and will add links as soon as you will add more pages.
Rest peter explains almost everything in detail like if you have URL issues and issues with crawling and indexing.
If you have a custom CMS, I think you should seriously consider the idea by Peter as this is something you need on regular basis anyways!
Hope this helps!
-
It's hard to tell without seeing your URL architecture.
First there are two specific terms and you never, never ever should forget them. They are - crawling and indexing. Once you prepare sitemap and submit there (or include in robots.txt) all bots get some map of your site and start crawling pages based on their crawling budget for your site. In crawling process they MAY find new pages that doesn't include in this map and will crawl them too. Again this is based on your crawling budget.
So when you submit sitemap - bot will get within seconds list of "non-crawled" 5000 pages and will start crawl them. Then he can find missed 500 pages and will crawl them too. Tricky is that when you update sitemap - he can detect quick changes there and start recrawling them again. But for missed 500 pages he can visit you again to check them for changes. And this will be also under your crawling budget. But if pages there isn't changed often - isn't big deal.
So you shouldn't hesitated about negative impact there. Only negative impact can happen if you have some serious URL architecture issues and messy URLs there. Then submitting partial sitemap can obfuscate this issues and some of your URLs to remain non-crawled.
Technically in SearchConsole you can see sitemap statistics like submitted and indexed. In perfect world numbers should be almost equal with little difference. But if you see huge difference between them - then you're in trouble. For example - on some site i have sitemap with submitted 44,950 pages and indexed of them was 29,643. This is pure example site crawling troubles or sitemap troubles. Because 1/3 of all pages isn't indexed at all.
PS: I forgot. You should use own CMS plugin for generating sitemap inside. Even if your CMS was custom made you should write (or hire someone) to create plugin inside. It's near 20-30 lines of write-here-your-favorite-language (PHP/Python/Perl/Ruby) and isn't big deal. This plugin will minimize crawling time from 3rd party sitemap generator tool because CMS already have all information inside and just need to be exported to XML.
-
It would definitely be better to submit a complete sitemap. If your site is built in Wordpress, Joomla, Magento, or many other standard CMS, it should have the ability to generate a full sitemap. Plugins like Yoast or Google Sitemaps help. Just depends on the site.
Otherwise you can probably get any pro SEO or agency to create a full 5500+ sitemap for you for $100 bucks or so. PM me if you need more help.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Should Google Trends Match Organic Traffic to My Site?
When looking at Google Trends and my Organic Traffic (using GA) as percentages of their total yearly values I have a correlation of .47. This correlation doesn't seem right when you consider that Google Trends (which is showing relative search traffic data) should match up pretty strongly to your Organic Traffic. Any thoughts on what might be going on? Why isn't Google Trends correlating with Organic Traffic? Shouldn't they be pulling from the same data set? Thanks, Jacob
Reporting & Analytics | | jacob.young.cricut0 -
Is it possible to set up one of the Goal Conversions on Google Analytics for a different site?
We are in the process of a website migration and need to set up the conversions for the new site. What is the most effective way of doing this?
Reporting & Analytics | | Sable_Group0 -
Curious, anyone ever had over half of their indexed links drop on an e-commerce site?
In a year went from around 300k indexed pages to around >100k according to GWT. Could this be duplicate content issue, lost links, spam, aged links or all of the above? either way an audit is in order. Thanks! Chris
Reporting & Analytics | | Sundance_Kidd0 -
Seomoz or GWMT correct?
I'm getting a few 404 errors show up in seomz, but a few thousand in gwmt. These are old site url's that may have been missed by redirects. Which one should I believe/attack first?
Reporting & Analytics | | Paul_MC0 -
Using Regex for Goals in Google Analytics
I’ve got a website with 20 different forms; and I’d like to track all 20 form completions with one Goal in Google Analytics. The completion-page URLs follow the pattern below (the website uses Drupal CMS) Form One: www.acme.com/node/2990/done?sid=651 Form Two: www.acme.com /node/2991/done?sid=785 Form Three: www.acme.com /node/2992/done?sid=1021 Form Four: www.acme.com /node/2993/done?sid=459 I believe that there is a way I can use regex so Google Analytics will track if any of these completion URLs appears. Looking for guidance how to format the regex statement.
Reporting & Analytics | | TopFloor0 -
Measuring events to external sites
Im having problem measuring click on ads by using events in GA or Jetpack. For example when I checked out yesterday this is what I read: 1. In GA events it says 12 clicks 2. In Jetpack it says 9 clicks But when I look at Referrals to the actual site directly it says 18 clicks Which one is the rights one? I need this because I use this to invoice clients end of month! and it cant be any "maybe".something. cheers, R
Reporting & Analytics | | rrrobertsson0 -
Site speed not being reported accurately?
We're constantly on the lookout for site speed, and Google's Webmaster tools are saying that we're really really slow (on the order of 5-15 seconds per page). But the site NEVER feels that slow, and lots of other tools say we're in the 3-5 second range. Further, we've implemented literally 100% of Google's suggestions, and all we have are ad units that now render using Googles Async ad loader, further reducing time to interactivity. Could Google be dinging us in search results for this? Here's an example page that they said loaded in 200+ seconds (!?!) http://hark.com/clips/kwkdqqtzsg-terran-nuclear-launch-detected Thanks!
Reporting & Analytics | | TheIronYuppie0 -
Overall site traffic - 3 quick questions
3 things : 1. Does Google factor in overall site traffic in rankings? So for 2 sites, all other things being pretty much equal, the one with higher traffic will be listed higher? 2. Does this logically imply that sites with lower traffic overall face an uphill struggle to be ranked highly??? 3. As an extension to this, would it be true to say that by increasing site traffic, say with Google Adwords or other online or offiline or whatever advertising, that might help get higher SEO rankings??? Thanks so much for your responses. This forum is great!
Reporting & Analytics | | inhouseninja0