Max Amout Of HTML Pages In A Folder
-
What's the maximum amount of html pages that one should put in a folder, to get the best SEO GoggleBot crawl? I'm aware that there's a limit of 10,000 on most servers, but was curious to know if a lesser amount of pages would be better, for crawling and indexing purposes. Also curious on peoples opinions on whether .jpg and .gif files should follow similiar rules.
-
Thanks for all the input. Google does seem to crawl everything these days, so I'm also in conclusion if the files fit, they'll get crawled. Sitemaps, internal links and optimized images are all a must.
-
For images, you want to make sure they're optimized for the web: small file sizes for easy download, but still a resolution that shows the image clearly. Your graphic designer and a good graphic design program (Photoshop, Gimp, etc.) should help with this.
-
Hi,
As Ray-pp said, there isn't an optimal number of pages that are going to serve you better.
However, if you want to help Google discover more about your site and pages of importances, look to create a good internal linking strategy. This doesn't mean that you should just add footer or sidebar links though - these are context links that talk about a different subject, along with a link to the appropriate page.
If you get this right, you can gain a lot in terms of Google understanding more about what you have to offer, and the links to primary pages can also lead to an increase in the SERPs for various phrases.
-Andy
-
AFAIK there is no efficient # of files to include in a folder directory for maximum crawl effectiveness. If you folder legitimately warrants 5k html pages in a directory, then Google will crawl all the pages. Make sure to create value-added pages with high quality content - Google will recognize them and crawl them as appropriate.
If you have the options, use your Google Webmaster Tools account to adjust crawl settings. Once your site is a specific size, Google will take-over crawl rate settings for you.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Noindex search pages?
Is it best to noindex search results pages, exclude them using robots.txt, or both?
Intermediate & Advanced SEO | | YairSpolter0 -
Can we mix up Wordpress and Html?
I have wordpress website. Now I am planning to add html service landing page into it. (by uploading html page on the root folder). After uploading it may look like below URL. My question is can I Html in wordpress website? Google like it or not? http://www.purpleflareagency.com/las-vegas-printing.html
Intermediate & Advanced SEO | | AlexanderWhite0 -
Home page not being indexed
Hi Moz crew. I have two sites (one is a client's and one is mine). They are both Wordpress sites and both are hosted on WP Engine. They have both been set up for a long time, and are "on-page" optimized. Pages from each site are indexed, but Google is not indexing the homepage for either site. Just to be clear - I can set up and work on a Wordpress site, but am not a programmer. Both seem to be fine according to my Moz dashboard. I have Webmaster tools set up for each - and as far as I can tell (definitely not an exper in webmaster tools) they are okay. I have done the obvious and checked that the the box preventing Google from crawling is not checked, and I believe I have set up the proper re-directs and canonicals.Thanks in advance! Brent
Intermediate & Advanced SEO | | EchelonSEO0 -
Google Webmaster Tools > HTML Improvements > 301 Moved Permanently pages - how did they even get there?
Hello experts! I'm going through my Google Webmaster Tools > HTML Improvements looking for pages with duplicate meta descriptions/titles that I can fix. And I noticed there are about 60 pages odd looking page titles that have duplicate meta descriptions, which are also noted as: 301 Moved Permanently Moved Permanently The document has moved here. Apache Server at sports When I click on the link to see the page names, all of them are pages we never created. The pages are all sports blog related. Here are few examples: http://www.titanium-jewelry.com/justin-tuck-blog.html http://www.titanium-jewelry.com/unlimited-potential-project-blog.html http://www.titanium-jewelry.com/left-handed-baseball-glove-blog.html http://www.titanium-jewelry.com/adjustable-basketball-hoops-blog.html how did they get on our site? Is this some sort of malicious attack? Most of them are sports related blog looking names. I just don't know how these pages could have been created. 2) is this hurting us with Google?3) Can you tell when the page was created?Thanks ron xEtX3op.jpg
Intermediate & Advanced SEO | | yatesandcojewelers0 -
Does Google make continued attempts to crawl an old page one it has followed a 301 to the new page?
I am curious about this for a couple of reasons. We have all dealt with a site who switched platforms and didn't plan properly and now have 1,000's of crawl errors. Many of the developers I have talked to have stated very clearly that the HTacccess file should not be used for 1,000's of singe redirects. I figured If I only needed them in their temporarily it wouldn't be an issue. I am curious if once Google follows a 301 from an old page to a new page, will they stop crawling the old page?
Intermediate & Advanced SEO | | RossFruin0 -
Are these doorway pages?
I've added category pages for counties/town on http://www.top-10-dating-reviews.com but will google see these as doorway pages? If you click on categories from the menu at the top and view some of the pages you'll hopefully see what I mean? Should I continue building these or delete them? Any advice appreciated.
Intermediate & Advanced SEO | | SamCUK0 -
Why are new pages not being indexed, and old pages (now in robots.txt) remain in the index?
I currently have a site that was recently restructured, causing much of its content to be reposted, creating new URL's for each page. To avoid duplicates, all of the existing pages were added to the robots file. That said, it has now been over a week - I know Google has recrawled the site - and when I search for term X, it is stil the old page that is ranking, with the new one nowhere to be seen. I'm assuming it's a cached version, but why are so many of the old pages still appearing in the index? Furthermore, all "tags" pages (it's a Q&A site, like this one) were also added to the robots a few months ago, yet I think they are all still appearing in the index. Anyone got any ideas about why this is happening, and how I can get my new pages indexed?
Intermediate & Advanced SEO | | corp08030 -
Not sure why Home page is outranked by less optimized internal pages.
We launched our website just three weeks ago, and one of our primary keyword phrases is "e-business consultants". Here's what I don't get. Our home page is the page most optimized around this search phrase. Using SEOmoz On-Page Optimization tool, the home page scores an "A". And yet it doesn't rank in the top 50 on Google Canada, although two other INTERNAL pages - www.ebusinessconsultants.ca/about/consulting-team/ & /www.ebusinessconsultants.ca/about/consulting-approach/ - rank 5 & 6 on Google Canada, even though they only score a grade "C" for on-page optimization for this keyword phrase. I've always understood that the home page is the most powerful page. Why are these others outranking it? I checked the crawl and Google Webmaster, and there is no obvious problem on the home page. Is this because the site is so new? It goes against all previous experience I've had in similar situation. Any guidance/ insight would be highly appreciated!!
Intermediate & Advanced SEO | | axelk0