WEBMASTER console: increase in the number of URLs we were blocked from crawling due to authorization permission errors.
-
Hi guys,I received this warning in my webmaster console: "Google detected a significant increase in the number of URLs we were blocked from crawling due to authorization permission errors." So i went to "Crawl Errors" section and i found such errors under "Access denied" status:
?page_name=Cheap+Viagra+Gold+Online&id=471
?page_name=Cheapest+Viagra+Us+Licensed+Pharmacies&id=1603
and many happy URLs like these. Does anybody know what this is and where it comes from?
Thanks in advance!
-
Thank you Tom!
-
Hi
to removed any chance of infection and I am not telling you that I am 100% sure it's infected
You must be certain that the regional infection was removed. If it was not and you had links created by a third party other than yourself you are better off getting it completely cleaned
use Sucuri.net to remove any chance of a hack.
Just type this into Google
- ?page_name=Cheap+Viagra+Gold+Online&id=471
- ?page_name=Cheapest+Viagra+Us+Licensed+Pharmacies&id=1603
http://www.pearsonified.com/2010/04/wordpress-pharma-hack.php
https://blog.sucuri.net/2010/07/understanding-and-cleaning-the-pharma-hack-on-wordpress.html
https://sitecheck.sucuri.net/results/www.davidandsonsjewelers.com/articles/author/carole/
i used deepcrawl.com to create the audit I you referenced.
&
Screaming frog SEO to create the site map
I hope that helps,
Tom
-
Hello Thomas,
I really appreciate your help! You said i can look at your site's structure. What is your site address?
Unfortunately, i still don't know what i need to do in order to remove those pharma hack from my site. If you know where to point me to get the answer, i'll be very grateful.
Also, what tool you used to generate this report http://crawl.blueprintmarketing.com/projects/reports/215533?ro=75ad0c6e4afacc428b553d449dfd281f82ec2ad6 ?
Also, what tool you used to create XML site map?
Thanks
-
No site map from checking multiple configurations of XML site maps and coming up with nothing no redirects either e.g. /sitemap_index.xml might exist separately or redirect to /sitemap.xml
http://www.davidandsonsjewelers.com/sitemap.xml shows a 404
Tool's
deepcrawl.com https://varvy.com/mobile/ & https://varvy.com/tools/
-
detect mobile issues
-
If I were you I would look at my site structure make sure that it was built in a certain manner for the right reasons.
If your traffic is all right you really do not want to change the site that much. If you do change the site change it slowly.
( A great example of this is how FireHost.com it is becoming Armor.com)
the tools I used to find out whether or not you had a site map primarily was deepcrawl.com
to detect mobile issues
https://varvy.com/mobile/ & https://varvy.com/tools/
http://i.imgur.com/W7BDaq7.png
http://www.screamingfrog.co.uk/seo-spider/
http://i.imgur.com/LbCBmmW.png
I used screaming frog to create a XML site map for you here
I would definitely add an XML site map.
Sincerely,
Thomas
-
Also, do you say that the mobile site is blocked? Also, how do you see that the site doesn't have XML? What tool shows you all this info?
Thanks
-
Hi Thomas,
I really appreciate your help! Can you advise me what i should do? I see all these reports but i don't know how i need to clean the site.
Thank you!
-
As you are showing certain URLs that are definitely Pharma hack their are certain things Sucuri is unable to detect because of it being a front-end tool not the PHP tool that would be needed for the two-part WordPress and PHP version of your site.
Just type this into Google
- ?page_name=Cheap+Viagra+Gold+Online&id=471
- ?page_name=Cheapest+Viagra+Us+Licensed+Pharmacies&id=1603
http://www.pearsonified.com/2010/04/wordpress-pharma-hack.php
https://blog.sucuri.net/2010/07/understanding-and-cleaning-the-pharma-hack-on-wordpress.html
https://sitecheck.sucuri.net/results/www.davidandsonsjewelers.com/articles/author/carole/
https://www.virustotal.com/en/ip-address/216.120.237.225/information/
http://dnsbl.inps.de/query.cgi?lang=en&ip=216.120.237.225&action=check&quick=0
-
and switch everything to WordPress
view-source:http://www.davidandsonsjewelers.com/
-
some of you are links are really not supposed to be there
Here is your report please use the URL below to navigate the entire report.
All of you are URLs are relative to the most part that should be fixed. You have a Java redirect that definitely needs to be fixed.
PDF & XML outline
- http://cl.ly/d6Sv/www.davidandsonsjewelers.com_http-www-davidandsonsjewelers-com-_13-09-2015_overview_215533.pdf
- http://cl.ly/d6S7/public-report_files-215533-www.davidandsonsjewelers.com_http-www-davidandsonsjewelers-com-_13-09-2015_overview_215533.xls
You have roughly 108 indexed URLs according to Google
https://marketing.grader.com/report/www.davidandsonsjewelers.com/overall
you do not have an XML site map unfortunately I found that out in the first five minutes but you can also find out if these things using
https://mza.seotoolninja.com/researchtools/crawl-test
upon a quick check with another tool I found
http://i.imgur.com/Y60WnIc.png
I love deepcrawl however your site is not large you can learn a lot about it with
http://www.screamingfrog.co.uk/seo-spider/ free
I hope this is a help, with analytics access and webmaster tool like this I cannot obviously give you a much better picture.
Tom
-
I will run the audit now sorry for the delay
-
-
The best way to solve this problem is to use
Or http://screamingfrog.co.uk Seo spider
If you give me the URL I will do it quick check for you.
-
Thank you Thomas,
My site is clean though according to sucuri. I spoke to owner of this website and they said that they were hacked in the past and they blocked those pages themselves. So now google detects those pages again? Or what exactly is happening? Anybody knows?
Thanks
-
Remember that not every URL is in Googles index. It does not mean that your back link is not in
https://mza.seotoolninja.com/researchtools/ose/
You should very quickly make sure that your website is not still completely full of malware like it sounds it is
use this tool to determined what has happened to your site if it is infected it is free.
If it is hacked as I believe it may be dependent on what you have described I would then purchase the malware removal and web application firewall
https://sucuri.net/website-antivirus/
if you would like a much more secure hosting environment https://armor.com is the best.
Once you have removed your site from the blacklists and removed all the bad where/malware make sure to crawl it with Google in Webmaster tools using fetch as a Google bot
your nightmare should be short-lived sorry to hear that your site was hacked hopefully this will get you back on track quickly.
-
Hi Dirk,
In webmaster tools if i click one by one those links, i can see "Linked from" URLs. There are URLs like this:
http://schwagginwagon.com/?page_name=Buying+Tadalis+SX+Safely+No+Prescription+Tadalis+SX&id=1810
and also there is one URL is coming from my domain. Not sure what it means.
I went through every single URL in Google index but all of them are normal URLs. Nothing related to spam. Any ideas?
Thanks
-
Try to do a search of type viagra site:yourdomain.com - and see if there are any pages of suspicious nature that are listed.
In the crawl error section in webmaster tools you could also check where these url's are coming from (external/internal links)
If your site is hacked - you can find more info here http://www.google.com/webmasters/hacked/ on what to do next.
rgds,
Dirk
-
Hello Dirk,
Thank you for fast reply! I thought it too right away. So all of these URLs are forbidden when i try to access them. This is the message from google webmaster tools "Googlebot couldn't crawl your URL because your server either requires authentication to access the page, or it is blocking Googlebot from accessing your site."
Any ideas? Thanks
-
Hi
On first sight I would guess your site has been hacked - do these url's exist when you try them?
Dirk
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search Console - Mobile Usability Errors
A site I'm looking at for a client had 100's of pages flagged as having Mobile Usability errors in Search Console. I found that the theme uses parameters in the URLs of some of theme resources (.js/.css) to identify the version strings. These were then being blocked by a rule in the robots.txt: "Disallow: /*?" I've removed this rule, and now when I inspect URLs and test the live versions of the page they are now being reported as mobile friendly. I then submitted validation requests in Search Console for both of the errors ("Text to small" and "Clickable Elements too close") My problem now, is that the validation has completed and the pages are still being reported as having the errors. I've double checked and they're find if I inspect them individually. Does anyone else have experience clearing these issues in Search Console? Any ideas what's going on here!
Technical SEO | | DougRoberts1 -
Very wierd pages. 2900 403 errors in page crawl for a site that only has 140 pages.
Hi there, I just made a crawl of the website of one of my clients with the crawl tool from moz. I have 2900 403 errors and there is only 140 pages on the website. I will give an exemple of what the crawl error gives me. | http://www.mysite.com/en/www.mysite.com/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | http://www.mysite.com/en/www.mysite.com/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/en/index.html#?lang=en | | | | | | | | | | There are 2900 pages like this. I have tried visiting the pages and they work, but they are only html pages without CSS. Can you guys help me to see what the problems is. We have experienced huge drops in traffic since Septembre.
Technical SEO | | H.M.N.0 -
Please Help! Crawl & Site Errors - Will This Impact My SEO?
Hello Moz, I need urgent help. I remove a tonne of product pages and put everything into one product page to deal with duplicate content. I thought this was a good thing to do until I got an email from Google saying: "Googlebot identified a significant increase in the number of URLs on ****.com that return a 404 (not found) error. " I checked it out and found the problem: 4 Soft 404's
Technical SEO | | crocman
41 Not Found's What do I need to do to fix this? Is it a problem or should I just ignore? I removed all the pages on WordPress but I need to do it somehow manually through Google? I have worked so hard on my SERP's that this will destroy me if I'm penalised. Please can someone advise?0 -
Why does my site have so many crawl errors relating to the wordpress login / captcha page
Going through the crawl of my site, there were around 100 medium priority issues, such as title element too short, and duplicate page title, and around 80 high priority issues relating to duplicate page content - However every page listed with these issues was the site's wordpress login / captcha page. Does anyone know how to resolve this?
Technical SEO | | ZenyaS0 -
How google crawls images and which url shows as source?
Hi, I noticed that some websites host their images to a different url than the one their actually website is hosted but in the end google link to the one that the site is hosted. Here is an example: This is a page of a hotel in booking.com: http://www.booking.com/hotel/us/harrah-s-caesars-palace.en-gb.html When I try a search for this hotel in google images it shows up one of the images of the slideshow. When I click on the image on Google search, if I choose the Visit Page button it links to the url above but the actual image is located in a totally different url: http://r-ec.bstatic.com/images/hotel/840x460/135/13526198.jpg My question is can you host your images to one site but show it to another site and in the end google will lead to the second one?
Technical SEO | | Tz_Seo0 -
Redirect_to in URLs?
I've never seen this before. I'm assuming that it's not SEO friendly and that these should be 301s or 302s instead? http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/cars-for-sale-search/results/central/riyadh/ford/explorer/2010/ford-explorer-2010-1038353 http://ksa-beta.motory.com/ar/login/?redirect_to=http://ksa-beta.motory.com/ar/account/my-saved-searches/
Technical SEO | | KatherineWatierOng0 -
First Crawl Report
Just joined SEOMoz today and am slightly overwhelmed, but excited about learning loads from it. I've just received my Crawl Report and there is a
Technical SEO | | iainmoran
404 : UserPreemptionError:
http://www.iainmoran.com/comments/feed/ This is a WordPress site and I've no idea what the best course of action to take. I've done some searching on Google and a couple of sites suggest removing that url from within the robots.txt file. I'm using the Yoast Plugin which apparently creates a robots.txt file, but I can't see any way to edit it. Is there another solution for resolving the 404 error? Many thanks, Iain.0 -
When should we use Remove URLs feature on Google Webmasters Tool?
Hi there, I run an ecommerce website on Magento. We are no longer using a category. It actually does not appear on the menu: mydomain.com/category.html If this is the case, do you recommend to remove it through the Removal URL feature on GWT? I don't want this to affect the juice of other links of the site such as: mydomain.com/product.html Thanks very much. Regards
Technical SEO | | footd0