Unsolved The Moz.com bot is overloading my server
-
. How to solve it?
-
maybe crawl delay will help.
-
@paulavervo
Hi,
We do! The best way to chat with us is via our contact form or direct email. We also have chat within Moz Pro.
Please contact us via [email protected] or https://mza.seotoolninja.com/help/contact
We will be happy to help.
Cheers,
Kerry. -
very nice brother i like it very good keep it up !
-
very nice !
-
does the moz team even monitor this forum?
-
If the Moz.com bot is overloading your server, there are several steps you can take to manage and mitigate the issue effectively. First, you can adjust the crawl rate in your
robots.txt
file by specifying a crawl delay for the Moz bot using directives likeUser-agent: rogerbot
andUser-agent: dotbot
, followed byCrawl-delay: 10
to make the bot wait 10 seconds between requests. If this does not suffice, you can temporarily block the bot by disallowing it in yourrobots.txt
file. Additionally, it's a good idea to contact Moz’s support team to explain the issue, as they may offer solutions to adjust the crawl rate for your site. Implementing server-side rate limiting is another effective strategy. For Apache servers, you can add rules in your.htaccess
file to return a 429 Too Many Requests status code to the Moz bots, while for Nginx servers, you can set up rate limiting in your configuration file to control the number of requests per second from a single user or IP address. Monitoring your server’s performance and log files can help identify specific patterns or peak times, allowing you to fine-tune your settings. Furthermore, using a Content Delivery Network (CDN) can help distribute the load by caching content and serving it from multiple locations, reducing the direct impact on your server caused by crawlers. By taking these steps, you can manage the load from the Moz.com bot and maintain your server’s stability and responsiveness.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Moz is showing a canonical error that dont belong.
Hi guys, and thanks for this excellent source of information. i have an issue with the moz system because is telling to me that i dont have canonical instructions but i have canonical instructions on all my pages, so... im confused because maybe im not understanding what the system want to show to me. if you can help me i will be very gratefull. here you can see a page that have the canonical instruction. https://drive.google.com/file/d/14U_-Sgu_NQaB7kMBH3AguHQMHyHX9L8X/view?usp=sharing and here you can see what is reporting to me the MOZ system. https://drive.google.com/file/d/1pqgSC-V9WOyBPvQEr06pbqpLf_w7-q8J/view?usp=sharing this is happening on 19 pages, and all the 19 pages have the canonical instruction.
On-Page Optimization | | b-lab
thanks in advance guys.0 -
Unsolved Why is'nt there an update yet!
0 -
Moz crawler is not able to crawl my website
Hi, i need help regarding Moz Can't Crawl Your Site i also share screenshot that Moz was unable to crawl your site on Mar 26, 2022. Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster.
Technical SEO | | JasonTorney
my robts.txt also ok i checked it
Here is my website https://whiskcreative.com.au
just check it please as soon as possibe0 -
Fluctuation in MOZ DA
HI, i want to know how MOZ DA increase or decrease. My Website: https://yesornogenerator.com/ has 8 DA according to MOZ but sometimes it goes back to 5 and then increase to 8. There is no change on backlinks so how this happens.
Moz Pro | | TOFHA0545150 -
How to quit moz 30days free?
dear moz: my english is so poor!!haha!! my user id si heroiamhero,my email is [email protected]. 2weeks ago i made a free on moz.com but i found it is not very useful for me ,so i want to quit free use ,please tell me how to quit moz 30days free,thankyou very much.
Moz Pro | | heroiamhero1 -
Why does SEO Moz say we are ranked lower than appears on Google Searches?
We are currently ranked 18th for the Sydney Vet keyword in SEO Moz ranking tools, however in organic Google search we are ranked third. This search was conducted without Googles personalised results feature. Is this just an error? Or does it have something to do with Google Places not being counted in SEO Moz ranking tools? Any help would be much appreciated.
Moz Pro | | Peter.Huxley590 -
Sudden decrease in Moz Page rank
Hello, We have a serious issue with 404s and recently saw our Moz Page rank fall from 53 to 47. 1. OSE Inbound links no longer shows any of our Linked In posts, did Linked In stop passing juice? 2. Does SEO Moz reduce your ranking when there's a sudden increase in 404s? 2a. WP Yoast SEO - I accidentally checked the box on this plugin to "Strip the category base (usually /category/) from the category URL" which basically caused all of our blog post categories and Datafeedr categories to disappear. Didn't realize till too much time had passed that I accidentally clicked that box. Datafeedr is a plugin for our estore that parses the data feeds from affiliate vendors and allows you to create a saved search that auto updates old products every 3 days. I had a no index/follow parameter on the category items, but seeing the # of 404s continue to increase, I temporarily removed this parm last week to see if it reduces this now static number of 404s. Google Webmaster tools started showing a ton of soft 404s that kept increasing, while SEO Moz didn't show any of those 404s. I didn't pay much attention to GWT since Google kept saying it won't affect our rankings, and nothing was showing up on SEO Moz. Last week a fraction of those 404s showed up and I am not sure if that's what lowered our Moz rank or what looks like a possible delinking from Linked In and a higher ranking complimentary website directly related to our field itsallaboutyoga. Looking at the Moz graph of "Total Linking Root Domains Over Time" all of our competitors took a similar % hit since between June and the end of July, so I am thinking its more wide based than fat fingered mistake. I fixed # 2, (have to still figure out what to do with most of those 404s, thinking of submitting a request to Google vs 1,000s of 301s) so in doing my review of this sequence of events and using it as a learning experience, where would I assign max destructive value as a percentage? A. Ignoring GWT soft 404s in favor of SEO Moz campaign reports B. No follow from Linked In and related industry site C. Datafeedr, thousands of indexed products through Datafeedr that are no longer available mostly due to WP Yoast SEO fat finger error. I did have the D. WP Yoast SEO, "Strip the category base (usually /category/) from the category URL" E. Global Google algo change Cheers, Michael
Moz Pro | | MKaloud0 -
SEO Moz ranking reports
I am confused. The SEO Moz ranking reports state that my site show show up in a specific position in SERP for a specific keyword yet when I google the keyword the ranking is different. Are these ranking reports merely indicative or are they supposed to be accurate?
Moz Pro | | casper4340