Www or no www
-
right now our organization has the a robot text file set to block search engines from indexing the non-www version of our URL - example.com
looks like this
#Dynamically generated robots.txt file for example.com
User-agent: *Disallow: /
Of course, we do allow search engines to crawl the www. version
Is this the best way to do this?
-
Likewise to you Simon, thanks a lot for the advice!
-
Thanks so much Kevin. Much appreciated!
-
I wouldn't recommend setting your robots.txt to disallow indexing of the site without www. The best method is to create a domain wide 301 redirect using .htaccess. You should also set it up in Google Webmaster tools under settings and select with www as your preferred domain. Below is an example of the rewrite in .htaccess.
RewriteEngine on
RewriteCond %{HTTP_HOST} ^mysite.com [NC]
RewriteRule (.*) http://www.mysite.com/$1 [L,R=301]
Hope that helps!
Kevin
-
Hi Kevin
That is a risk, you could end up blocking search engines from indexing your entire website. Usually, both www and non-www point to the same location, so one robots.txt file can apply to both (unless you've configured your site differently).
Check out http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=44231
I'd suggest removing that Disallow command within your robots.txt file and do the following:
Within Google Webmaster Tools, go to 'Site configuration' 'Settings' ~ from there you can set how you'd like your pages to be displayed in Google's search results, either with or without the www. (or can leave Google to decide).
Regards
Simon
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Please who can help me rank my website www.shopdocuments.com ?
Hi , i bought my website this month , i built it myself now i want it to grow seo on google pages and stuff tried google ads but they always block my account don't know why so somebody help me please any company? individual willing to work it for me ? even if paid company no problem
Technical SEO | | planetdocs1 -
Www to non www on a .com/blog url
hi guys, I have had to reset my site from www to non-www. via htacces and this worked out just fine.However, the /blog WordPress section will not redirect to the non-www. I have changed the config.php to non-www. However, the /blog WordPress section will not redirect to the non-www. I have changed the config.php to non-www. Does anyone have an idea as to what I need to do to force the non-www in a folder installed blog http://5starweddingdirectory.com/ http://www.5starweddingdirectory.com/blog/ Regards T
Technical SEO | | Taiger0 -
Best Practice for www and non www
How is the best way to handle all the different variations of a website in terms of www | non www | http | https? In Google Search Console, I have all 4 versions and I have selected a preference. In Open Site Explorer I can see that the www and non www versions are treated differently with one group of links pointing to each version of the same page. This gives a different PA score. eg. http://mydomain.com DA 25 PA 35 http://www.mydomain.com DA 19 PA 21 Each version of the home page having it's only set of links and scores. Should I try and "consolidate" all the scores into one page? Should I set up redirects to my preferred version of the website? Thanks in advance
Technical SEO | | I.AM.Strategist0 -
Duplicate Content Showing up on Moz Crawl | www. vs. no-www.
Hello Moz Community! I am new to SEO, Moz and this is my first question. My questions; I have a client that is getting flagged for Duplicate Content. He is getting flagged for having two domains that have the same content i.e. www.mysite.com & mysite.com. I read into this and set up a 301 redirect through my hosting site. I evaluated which site had a stronger Page Authority and had the weaker site redirect to the stronger site. However, I am still getting hit for Duplicate pages caused by the www.mysite.com & mysite.com being duplicates. How should I go about resolving this? Is this an example of a Canonical tag needed in the head of the HTML? Any direction is appreciated. Thank You. B/R Will H.
Technical SEO | | MarketingChimp100 -
Which to redirect to, www or non-www?
Please help. I have a client who's site has duplicate content issues due mainly to the www/non-www problem. The MOZ page authority of both is the same, the www version has more linking root domains, and for most of the keywords the www version shows up in Google. The developer indicated the site is on the non-www. I need to do a .htacess 301 redirect to eliminate the dup content problems. Which do I redirect to? Any help is greatly appreciated, Thanks!!
Technical SEO | | SteveFaber0 -
Redirecting non-www to www
Hi all, I recently ran my first diagnostic test with SEOmoz and was alarmed to find my company's site has over 8,000 cases of duplicate content, virtually all of which can be attributed to separate domains, www vs. non-www. So after some research I found that this can be solved easily using .htaccess. However I found a warning on another site that if my site has already been indexed by Google without the www, there could be side effects like a loss in PR. Can anybody tell me how to find out whether my site falls into this category? I do have access to Google Webmaster tools but I can't find anywhere that tells me how my site's been indexed. Thanks in advance.
Technical SEO | | rylaughlin0 -
Www to non www
Hi, Is this the correct code for redirecting www to non www version on Apache server RewriteEngine OnRewriteCond %{HTTP_HOST} ^www.example.com RewriteRule (.*) http://www.example.com/$1 [R=301,L] Thanks
Technical SEO | | seoug_20050 -
Root domain not resolving to www. Duplicate content?
Hi, I'm working with a domain that stays on the root domain if the www is not included. But if the www is included, it stays with the www. LIke this: example.com
Technical SEO | | HardyIntl
or
www.example.com Of course, they are identical and both go to the same IP. Do search engines consider that to be duplicate content? thanks,
michael0