Magento Robots & overly dynamic URL-s
-
How can i block all URL-s on a Magento store that have 2 or more dynamic parameters in it, since all the parameters have attribute name in it and not some uniform ID
Would something like:
Disallow: /?&*
work? Since the only thing that is constant throughout all the custom parameters is that they are separated with "&"
Thanks
-
Thanks, it worked
-
Your code should work, add it then test a dynamic url via GWT > Health > Blocked URLs
You can also set custom parameters to either be tracked or not tracked via GWT > Configuration > URL Parameters
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Dynamic Url best approach
Hi We are currently in the process of making changes to our travel site where by if someone does a search this information can be stored and also if the user needs to can take the URL and paste into their browser at find that search again. The url will be dynamic for every search, so in order to stop duplicate content I wanted ask what would be the best approach to create the URLS. ** An example of the URL is: ** package-search/holidays/hotelFilters/?depart=LGW&arrival=BJV&sdate=20150812&edate=20150819&adult=2&child=0&infant=0&fsearch=first&directf=false&nights=7&tsdate=&rooms=1&r1a=2&r1c=0&r1i=0&&dest=3&desid=1&rating=&htype=all&btype=all&filter=no&page=1 I wanted to know if people have previous experience in something like this and what would be the best option for SEO. Will we need to create the URL with a # ( As i read this stops google crawling after the #) Block the folder IN ROBOTS is there any other areas I should be aware of in order stop any duplicate content and 404 pages once the URL/HOLIDAY SEARCH is no longer valid. thanks E
Technical SEO | | Direct_Ram0 -
Conversion of URL's for Readability
Reading over Rands latest Post about URL structure I had a quick question about the best way to convert URL's that don't have perfect URL structure... Current the Structure of our E-commerce store has a structure that is not friendly with domain.com/product/zdcd-jobd3d-fdoh what is the easiest way to convert these to read URL's without causing any disruptions with the SERP. Are we talking about a MOD-Rewrite in the CMS.......
Technical SEO | | CMcMullen0 -
Do I have a robots.txt problem?
I have the little yellow exclamation point under my robots.txt fetch as you can see here- http://imgur.com/wuWdtvO This version shows no errors or warnings- http://imgur.com/uqbmbug Under the tester I can currently see the latest version. This site hasn't changed URLs recently, and we haven't made any changes to the robots.txt file for two years. This problem just started in the last month. Should I worry?
Technical SEO | | EcommerceSite0 -
HTTP & HTTPS
what is best recommended when some of the pages on site goes from HTTP to HTTPS: 301 redirection or 302 redirection?
Technical SEO | | JonsonSwartz
and why? thank you I was asked to elaborate so: on my website I have open account pages. users are asked to fill the details. those page are secured and are HTTPS. the problem is that the whole website turned to HTTPS so they redirected most of the pages from HTTPS to HTTP.
the secured pages are redirected from HTTP to HTTPS. I wanted to check if it's correct and what is the best redirection way (301 or 302)0 -
Robots.txt file
How do i get Google to stop indexing my old pages and start indexing my new pages even months down the line? Do i need to install a Robots.txt file on each page?
Technical SEO | | gimes0 -
Structuring URL's for better SEO
Hello, We were rolling our fresh urls for our new service website. Currently we have our structure as www.practo.com/health/dental/clinic/bangalore We like to have it as www.practo.com/health/dental-clinic-bangalore Can someone advice us better which one of the above structure would work out better and why? Should this be a focus of attention while going ahead since this is like a search engine platform for patients looking out for actual doctors. Thanks, Aditya
Technical SEO | | shanky10 -
Invisible robots.txt?
So here's a weird one... Client comes to me for some simple changes, turns out there are some major issues with the site, one of which is that none of the correct content pages are showing up in Google, just ancillary (outdated) ones. Looks like an issue because even the main homepage isn't showing up with a "site:domain.com" So, I add to Webmaster Tools and, after an hour or so, I get the red bar of doom, "robots.txt is blocking important pages." I check it out in Webmasters and, sure enough, it's a "User agent: * Disallow /" ACK! But wait... there's no robots.txt to be found on the server. I can go to domain.com/robots.txt and see it but nothing via FTP. I upload a new one and, thankfully, that is now showing but I've never seen that before. Question is: can a robots.txt file be stored in a way that can't be seen? Thanks!
Technical SEO | | joshcanhelp0 -
Removing duplicate &var=1 etc var name urls from google
Hi I had a huge drop in traffic around the 11th of july over 50% down with no recovery as yet... ~5000 organic visits per day down to barley over 2500. I fixed up a problem that one script was introducing that had caused high bounce rates. Now i have identified that google has indexed the entire news section 4 times, same content but with var=0 var=1 2 3 etc around 40,000 urls in total. Now this would have to be causing problems. I have fixed the problem and those url's 404 now, no need for 301's as they are not linked to from anywhere. How can I get them out of the index? I cant do it one by one with the url removal request.. I cant remove a directory from url removal tool as the reuglar content is still there.. If I ban it in robots.txt those urls, wont it never try to index them again and thus not ever discover they are 404ing? These urls are no longer linked to from anywhere, so how can google ever reach them by crawling to find them 404ing?
Technical SEO | | Adsau0