How to Set Custom Crawl Rate in Google Webmaster Tools?
-
This is really silly question to set custom crawl rate in Google webmaster tools. Any one can find out that section under setting tab. But, I have confusion to decide number for request per second and second between requests text field.
I want to set custom crawl rate for my eCommerce website.
I checked my Google webmaster tools and find out as attachment. So, Can I use this facility to improve my crawling?
-
Hi Daniel,
I was thinking about it due to very less number of indexing. Thanks for your suggestion.
-
It's better to just leave it on default - custom crawl rates are only needed if you're experiencing traffic problems from the server(as stated in your screenshot).
Yyou will also use more bandwidth on your own server by the google bot coming more often so this should also be something you think about. Also worth mentioning is that after 90 days it will go back to default settings.
http://googlewebmastercentral.blogspot.com/2006/10/learn-more-about-googlebots-crawl-of.html
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google image search
How does google decide which image show up in the image search section ? Is is based on the alt tag of the image or is google able to detect what is image is about using neural nets ? If it is using neural nets are the images you put on your website taken into account to rank a page ? Let's say I do walking tours in Italy and put a picture of the leaning tower of pisa as a top image while I be penalised because even though the picture is in italy, you don't see anyone walking ? Thank you,
Intermediate & Advanced SEO | | seoanalytics1 -
"Null" appearing as top keyword in "Content Keywords" under Google index in Google Search Console
Hi, "Null" is appearing as top keyword in Google search console > Google Index > Content Keywords for our site http://goo.gl/cKaQ4K . We do not use "null" as keyword on site. We are not able to find why Google is treating "null" as a keyword for our site. Is anyone facing such issue. Thanks & Regards
Intermediate & Advanced SEO | | vivekrathore0 -
URL Errors for SmartPhone in Google Search Console/Webmaster Tools
Howdy all, In recent weeks I have seen a steady increase in the number of smartphone related url errors on Googles Search Console (formerly webmaster tools). THe crawler appears to be searching for a /m/ or /mobile/ directory within the URLs. Why is it doing this? Any insight would be greatly appreciated. Unfortunately this is for an unresponsive site, would setting the viewport help stop the issue for know until my new responsive site is launched shortly. Cheers fello Mozzers 🙂 Tim NDh1RNs
Intermediate & Advanced SEO | | TimHolmes1 -
Google News and Meta Title
Hi,
Intermediate & Advanced SEO | | JohnPalmer
1. I just read this article: https://www.seroundtable.com/google-news-titles-h1-19876.html
Google want the same title. no problem. but what about the brand? for example
POST TITLE BLA BLU | My Brand
The "post title bla blu" is the H1 and title of the article and | My Brand is my brand...
I can keep it as is with the My brand? or remove it? what about posts with long title for example "POST TITLE BLA BLU POST TITLE BLA BLU | My Brand"
What is you suggestion, I know Google doesn't show all the text and we'll see "...". it's still important to write the brand name in the title or just the post title? (without the brand). Thanks,0 -
Improve Response Rate
Hi fellow Mozzers, I have recently been trying to earn new links for my company websites by appealing to local public libraries that line up with our geographic footprint. We have store locations in 569 city, state combinations, so I am finding the local library for each city, state and asking them to add us to their website as a community reference for child care. Here is my pitch email that I am sending out.... Hi There, _ I hope this email finds you in good health and good spirits. _ My name is XXXXX and I represent <insert company="" name="">. I am reaching out to local libraries in the cities and counties where our schools are located in an effort to request local support for our schools. Many libraries across the country have reached out to us asking for parent resources and child care options in their local city. I would like to extend the same value to you.</insert> _ I would love to have our school in <insert city="" name="">be considered for placement on your public library site found at <insert target="" website="" url="">. With ample parent resources and child related materials, our school would be a great fit for your local community. We have many great parenting articles, blogs and local child care options in <insert city="" name="">.</insert></insert></insert>_ Our local school website can be found at : Our parent resource section can be found at: Who do you suggest I speak with regarding earning a placement on your site? Any information would be greatly appreciated. So far I have a response rate of about 1% and of those that do respond, none are willing to add a link on their site in any capacity since our company is privately held and not a public school option. Can anyone suggest either a better way to approach link earning on a city by city basis or help me improve my current initial email to get a better response? I really appreciate it! p.s.... some of my other link targets are city run websites, local chamber of commerce chapters (suspect this will be hard since most require a paid membership), local public school sites, and keyword targeted lists such as "child care in phoenix, az" to see what sites come for each term. Advice on these and anything I may be missing would be of great value.
Intermediate & Advanced SEO | | dsinger0 -
Crawling issue
Hello, I am working on 3 weeks old new Magento website. On GWT, under index status >advanced, I can only see 1 crawl on the 4th day of launching and I don't see any numbers for indexed or blocked status. | Total indexed | Ever crawled | Blocked by robots | Removed |
Intermediate & Advanced SEO | | sedamiran
| 0 | 1 | 0 | 0 | I can see the traffic on Google Analytic and i can see the website on SERPS when i search for some of the keywords, i can see the links appear on Google but i don't see any numbers on GWT.. As far as I check there is no 'no index' or robot block issue but Google doesn't crawl the website for some reason. Any ideas why i cannot see any numbers for indexed or crawled status on GWT? Thanks Seda | | | | |
| | | | |0 -
Dropped Out of Google and Bing
I am helping with a site that at one time I had on page 1 for Google/Bing. Site started to slip in rankings, then someone else did a makeover of the store and botched things by renaming pages, having errors in pages (multiple head/body), mismatch page names from sitemap, etc. Site slipped to page 4/5. I righted things, fixed duplication using canonicalization, made some other changes. Now site is gone completely from Google/Bing for desired keyword. No penalties. Site still shows if do search on domain name. Site is www.plussizeplum.com (plus size lingerie, sorry), keyword target is plus size lingerie. Anyone have any clues, tips, etc on why we fell off the face of the earth? Page Authority/Domain Authority are both comparable to most of the page 1/2 sites for same thing. Thanks for any advice.
Intermediate & Advanced SEO | | dlcohen0 -
Old deleted sitemap still shown in webmaster tools
Hello I have redisgned a website inl new url structure in cms. Old sitemap was not set to 404 but changed with new sitemap files,also new sitemap was named different to old one.All redirections done properly Still 3 month after google still shows me duplicate titile and metas by comparing old and new urls I am lost in what to do now to eliminate the shown error. How can google show urls that are not shown in sitemap any more? Looking forward to any help Michelles
Intermediate & Advanced SEO | | Tit0