Site Crawling with Firewall Plugin
-
Just wondering if anyone has any experience with the WordPress Simple Firewall plugin. I have a client who is concerned about security as they've had issues in that realm in the past and they've since installed this plugin: https://wordpress.org/support/view/plugin-reviews/wp-simple-firewall?filter=4
Problem is, even with a proper robots file and appropriate settings within the firewall, I still cannot crawl the site with site crawler tools.
Google seems to be accessing the site fine, but I still wonder if it is in anyway potentially hindering search spiders.
-
I would contact the developer of the plugin for those types of questions. He seems very responsive to people's input on the discussion section of his plugin. They do serve practical uses though, like preventing comment spam and such, plus your client has been specifically targeted in the past so it seems it is a value add moving forward.
-
Thanks Ryan. I guess my main concern is not so much running crawlers on the site, but making sure that search engine spiders have completely unrestricted access. In the settings for this plugin you can tell it to "try to ignore search bots". The word try, doesn't really sound all that reassuring to me. Are these type of firewalls even effective at prohibiting attacks? I'd rather just get rid of the plugin all together if the risk vs reward doesn't add up.
-
Hi Jay. You might have to disable this plugin when crawling the site as it would also create a heavier load than usual when crawling with it on (even if you are able to crawl while it's on). So a possible interim solution is disable at a low traffic time of day, crawl, then re-enable. Cheers!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301'd site, but new site is not getting picked up in google.
Hi I'm having big issues! Any help would be greatly appreciated This is the 3rd time this happened. Every time I switch my old site greatcleanjokes.com to the new design of chokeonajoke.com traffic goes almost completely down (I even tried out the new design on greatcleanjokes [to see if it was a 301 issue] and traffic also went down.) What can possibly be wrong with this new site that google just doesn't like it ?! I was ranking high up for many big phrase like joke of the day, corny jokes, clean jokes, short jokes. Now It's all gone. I also think it's strange that when I search for site:chokeonajoke.com the post pages show up before the category pages!? Here is the old site http://web.archive.org/web/20140406214615/http://www.greatcleanjokes.com/ Here is the new one http://chokeonajoke.com/ If you can't figure out anything do you know of anyone I can hire who may be able to figure it out?
Technical SEO | | Nickys22111 -
Site not loading on Firefox
Hello guys, I can't get my website to be loaded on Firefox, why's that?
Technical SEO | | PremioOscar0 -
Our stage site got crawled and we got an unnatural inbound links warning. What now?
live site: www.mybarnwoodframes.com stage site: www.methodseo.net We recently finished a redesign of our site to improve our navigation. Our developer insisted on hosting the stage site on her own server with a separate domain while she worked on it. However, somebody left the site turned on one day and Google crawled the entire thing. Now we have 4,320 pages of 100% identical duplicate content with this other site. We were upset but didn't think that it would have any serious repercussions until we got two orders from customers from the stage site one day. Turns out that the second site was ranking pretty decently for a duplicate site with 0 links, the worst was yet to come however. During the 3 months of the redesign our rankings on our live site dropped and we suffered a 60% drop in organic search traffic. On May 22, 2013 day of the Penguin 2.0 release we received an unnatural inbound links warning. Google webmaster tools shows 4,320 of our 8,000 links coming from the stage site domain to our live site, we figure that was the cause of the warning. We finished the redesign around May 14th and we took down the stage site, but it is still showing up in the search results and the 4,320 links are still showing up in our webmaster tools. 1. Are we correct to assume that it was the stage site that caused the unnatural links warning? 2. Do you think that it was the stage site that caused the drop in traffic? After doing a link audit I can't find any large amount of horrendously bad links coming to the site. 3. Now that the stage site has been taken down, how do we get it out of Google's indexes? Will it be taken out over time or do we need to do something on our end for it to be delisted? 4. Once it's delisted the links coming from it should go away, in the meantime however, should we disavow all of the links from the stage site? Do we need to file a reconsideration request or should we just be patient and let them go away naturally? 5. Do you think that our rankings will ever recover?
Technical SEO | | gallreddy0 -
Joomla Title Help - Plugin?
I am currently working on a site in Joomla (2.5) and continue to run across issues with titles. Currently, the site is set to display the page title just over the content. Turning this off really is not an option, as it is necessary to keep these titles as part of the content. Right now, my page titles are not well optimized. What I would like to have is a separate meta title used for SEO purposes, similar to what a plugin like Yoast would do in WordPress - where there is a name for a page (often used in content as well), but a meta title that is used by Google. Hope that makes sense. If anyone knows of a plugin that may be helpful, that would be great. Thanks!
Technical SEO | | DeliaAssociates0 -
Should I show archives on site?
Should I show my archives on my site? I have WordPress and have dragged the archive widget to the bottom. Would this be considered duplicate content?This is what it looks like. October 2012 September 2012 August 2012 July 2012 June 2012 May 2012 April 2012 February 2012
Technical SEO | | MyAllenMedia0 -
International Site Links In Footer
We have several international sites and we have them linked in the footer of our main .com site . Should we add "nofollow" to these links? Our concern is that Google could see these sites as a network?
Technical SEO | | EwanFisher0 -
How do i use Linkedin to promote my site
Hi i have heard so much about Linkedin but i am not sure how to use it. I would like to start off using it for free and have opened an account but i am not sure how to use it to promote my site and to gain traffic before i go to the paid service. Can anyone please give me some basic advice on how to use the free service to gain traffic to my site and also the best way to use it. I want to try and use it to promote my articles on www.in2town.co.uk
Technical SEO | | ClaireH-1848860 -
Site Structure question
when deciding the Site structure for a e-commerce site Is it better to keep everything mysite.com/widget.html or use categories like mysite.com/Gifts/widget.html
Technical SEO | | DavidKonigsberg0