How best to fix 301 redirect problems
-
Hi all
Wondering if anyone could help out with this one. Roger Bot crawler has just performed it's weekly error crawl on my site and I appear to have 18,613 temp redirect problems!! Rather, the same 1 problem 18,613 times.
My site is a magento store and the errors it is giving me is due to the wishlist feature on the site. For example, it is trying to crawl links such as index.php/wishlist/index/add/product/29416/form_key/DBDSNAJOfP2YGgfW (which would normally add the item to one's wishlist). However, because Roger isn't logged into the website it means that all these requests are being sent to the login url with the page title of Please Enable Cookies.
Would the best way to fix this be to enable wishlists for guests? I would rather not do that but cannot think of another way of fixing it.
Any other Magento people come across this issue?
Thanks, Carl
-
thanks. I completely forgot about the robots.txt method of fixing it. Will give it a go. I will add comparison in there too just to be safe.
-
Hey Carl
I have this problem a lot on my Magento stores since theres a ton of ways to access the same product and it can be an SEO nightmare. If you have FTP access I would recommend adding this to the robots.txt file:
Disallow: /wishlist/
This will stop the bots from crawling everything passed /wishlist/. Here's a good resource if you need further assistance: http://www.magentocommerce.com/knowledge-base/entry/setting-up-a-robotstxt-file
Hopefully that helps!
Good luck
-
Thanks will try and work out how to do that. Magento is a pain to make minor changes to.
Hopefully the crawl stats will look a little healthier next week !!
-
Use Rel="no index, no follow" on the link and roger will ignore the link. Job done
You wouldn't get any advantage of seeing the added items in someones wishlist as it would be duplicate content I'm presuming.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 Redirects, Sitemaps and Indexing - How to hide redirected urls from search engines?
We have several pages in our site like this one, http://www.spectralink.com/solutions, which redirect to deeper page, http://www.spectralink.com/solutions/work-smarter-not-harder. Both urls are listed in the sitemap and both pages are being indexed. Should we remove those redirecting pages from the site map? Should we prevent the redirecting url from being indexed? If so, what's the best way to do that?
Technical SEO | | HeroDesignStudio0 -
I have an eCommerce store with a lot of 301 redirects. Would that hurt my rankings?
I use BigCommerce, and they have a system where all the URLs are dynamically generated from the name of each product. So if I named a product "widget x y z" the url would be /widget-x-y-z/, and if I changed that to "blue widget x y z", it would change to /blue-widget-x-y-z/ and automatically redirect the old one to the new one. As a result, in 6 months, because of a lot of tweaking and experimenting, I've ended up with a hefty list of 400 redirects. Some of them are very old, and some are recent. So my question is in two parts: a) does having all of these redirects hurt my rankings? b) if so, would deleting them help?
Technical SEO | | shabbirun0 -
301 redirects - one overall redirect or an individual one for each page url
Hi I am working on a site that is to relaunch later on this year - is best practise for the old urls (of which there are thousands) to write a piece of code that will cover all of the urls and redirect them to the new home page or to individually redirect each url to its new counterpart on the new site. I am naturally concerned about user experience on this plus losing our Google love we currently have but am aware of the time it would take to do this individually. Any advice would be appreciated. Thanks
Technical SEO | | Pday1 -
301 Redirects on Large Real Estate Website
Hi guys,We are about to move over to a new website and need advice on handling the 301 redirects.We have a large real estate website with around 12,000 pages, a lot of these are properties (about 10,000)On our old website, the url structure for each property is as follows -domainname.com/property/view?property=14863on our new site, the url structure isdomainname.com/properties/view/6137The property ID number is always different from old site to new. The way we see it, we have two options. a.) a manual redirect of each and every property url. A very very long jobb.) a folder level redirect, so redirect the 'property' folder on the old site into the 'properties' folder on new. The con with this one is we are not sure if this is the best route to take, if it is how we would go about it?Some advice would be really appreciated guys. I know there are some hyper intelligent SEO's in here and we need to make sure we handle this right!Many thanks in advance.Mark
Technical SEO | | Nextman0 -
302 or 301 redirect to https ?
I am redirecting whole site to https. Is there a difference between 302 or 301 redirect for seo? Site never been indexed. Planning to do that with .htaccess command RewriteCond %{HTTPS} !=on
Technical SEO | | Kotkov
RewriteRule ^(.*) https://%{SERVER_NAME}/$1 [R,L] There are plenty of ways http://www.askapache.com/htaccess/ssl-example-usage-in-htaccess.html Which way would be the best? Thanks is advance0 -
Looking for some help adding a 301 redirect for my Site
Hi there, I am trying to eliminate the 'www' using a 301 redirect script as I have duplicate page titles for both versions (with and without the 'www') I checked the page authority and found the pages without the 'www' to be ranked higher. For this reason I believe it would be wise to go for this option. I have an .htaccess file, all I need is the code and I should be ok 🙂 Thanks!
Technical SEO | | debeenus0 -
301 Redirect for homepage with language code
In my multilingual Magento store, I want to redirect the hompage URL with an added language code to the base URL. For example, I want to redirect http://www.mysite.com/tw/ to http://www.mysite.com/ which has the exact same content. Using a canonical URL will help with search engines, but I would just rather nip the problem in the butt by not showing http://www.mysite.com/tw/ to visitors in the first place. Problem is that I don't want (can't have) all /tw/ removed from URLs due to Magento limitations, so I just want to know how to redirect this single URL. Since rewrites are on, adding Redirect 301 /tw http://www.88kbbq.com would redirect all URLs with the /tw/ language code to ones without. Not an option. Hope folks can lend a hand here.
Technical SEO | | kwoolf0 -
Accidently did a 301 redirect on root domain and lost domain keyword position
I just bought a domain about a week ago and instantly ranked number 4 for for my keywords with the domain keyword bonus. I created a landing page off the root of my domain while I'm building out my main site. I accidentally did a 301 redirect instead of a 302 from my root to my landing paging and this resulted in me losing my position and only being about to find my domain in the google if I searched for my domain specifically. Anyway to regain my original position? I have removed the redirect. Have I been put in the sandbox?
Technical SEO | | JohnTurner790