Pitfalls when implementing the “VARY User-Agent” server response
-
We serve up different desktop/mobile optimized html on the same URL, based on a visitor’s device type.
While Google continue to recommend the HTTP Vary: User-Agent header for mobile specific versions of the page (http://www.youtube.com/watch?v=va6qtaiZRHg), we’re also aware of issues raised around CDN caching; http://searchengineland.com/mobile-site-configuration-the-varies-header-for-enterprise-seo-163004 / http://searchenginewatch.com/article/2249533/How-Googles-Mobile-Best-Practices-Can-Slow-Your-Site-Down / http://orcaman.blogspot.com/2013/08/cdn-caching-problems-vary-user-agent.html
As this is primarily for Google's benefit, it's been proposed that we only returning the Vary: User-Agent header when a Google user agent is detected (Googlebot/MobileBot/AdBot).
So here's the thing: as the server header response is not “content” per se I think this could be an okay solution, though wanted to throw it out there to the esteemed Moz community and get some additional feedback.
You guys see any issues/problems with implementing this solution?
Cheers!
linklater
-
So, there are lots of 'ifs' here, but the primary problem I see with your plan is that the CDN will return the content to Googlebot without the request hitting your server so you won't have the option to serve different headers to Googlebot.
Remember that every page is the main HTML content (which may be static or dynamically generated for every request), and then a whole bunch of other resources (Javascript and CSS files, images, font files etc.). These other resources are typically static and lend themselves far better to being cached.
Are your pages static or dynamic? If they are dynamic then you are possibly not benefitting from them being cached anyway, so you could use the 'vary' header on just these pages, and not on any static resources. This would ensure your static resources are cached by your CDN and give you a lot of the benefit of the CDN, and only the dynamic HTML content is served directly from the server.
If most of your pages are static you could still use this approach, but just without the full benefit of the CDN, which sucks.
Some of the CDNs are already working on this (see http://www.computerworld.com/s/article/9225343/Akamai_eyes_acceleration_boost_for_mobile_content and http://orcaman.blogspot.co.uk/2013/08/cdn-caching-problems-vary-user-agent.html) to try and find better solutions.
I hope some of this helps!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Homepage organization schema question: logo lives on amazon server, can I call that out on the structured data?
Basically, the homepage organization schema has called out the logo, but it lives on the amazon server. We're having issues with Google rendering the correct logo on the knowledge graph. The URL for the amazon asset looks something like this: <brandname>-assets.s3-us-west-2.amazonaws.com/<logo>.png</logo></brandname> Calling that out on the organization structured data for the logo is okay right?
Intermediate & Advanced SEO | | imjonny1230 -
Pages excluded from Google's index due to "different canonicalization than user"
Hi MOZ community, A few weeks ago we noticed a complete collapse in traffic on some of our pages (7 out of around 150 blog posts in question). We were able to confirm that those pages disappeared for good from Google's index at the end of January '18, they were still findable via all other major search engines. Using Google's Search Console (previously Webmastertools) we found the unindexed URLs in the list of pages being excluded because "Google chose different canonical than user". Content-wise, the page that Google falsely determines as canonical instead has little to no similarity to the pages it thereby excludes from the index. False canonicalization About our setup: We are a SPA, delivering our pages pre-rendered, each with an (empty) rel=canonical tag in the HTTP header that's then dynamically filled with a self-referential link to the pages own URL via Javascript. This seemed and seems to work fine for 99% of our pages but happens to fail for one of our top performing ones (which is why the hassle 😉 ). What we tried so far: going through every step of this handy guide: https://mza.seotoolninja.com/blog/panic-stations-how-to-handle-an-important-page-disappearing-from-google-case-study --> inconclusive (healthy pages, no penalties etc.) manually requesting re-indexation via Search Console --> immediately brought back some pages, others shortly re-appeared in the index then got kicked again for the aforementioned reasons checking other search engines --> pages are only gone from Google, can still be found via Bing, DuckDuckGo and other search engines Questions to you: How does the Googlebot operate with Javascript and does anybody know if their setup has changed in that respect around the end of January? Could you think of any other reason to cause the behavior described above? Eternally thankful for any help! ldWB9
Intermediate & Advanced SEO | | SvenRi1 -
How to Implement AMP for Single Blog Post?
Hello Moz Team, I would like to implement AMP for my single blog post not on whole blog. Is it possible? if Yes then How? Note - I am already using GTM for my website abcd.com but I would like to use for my blog post only and my blog is like - abcd.com/blog..............let me clarify Blog Post means - abcd.com/blog/my-favorite-dress Thanks!
Intermediate & Advanced SEO | | Johny123450 -
Benefits of Implementing Open Graph?
I understand the benefits of implementing Open Graph as a local business, but am unclear of the benefits for a website, for example, a health website. Does Bing use OG in their search algorithm?
Intermediate & Advanced SEO | | nicole.healthline0 -
Server Migration, Does it effect SEO?
About to go through a server migration. My intitial thought is that a change in servers shouldn't really change my rankings. But I've heard rumors... Can a server migration change rankings? Why?
Intermediate & Advanced SEO | | Thos0030 -
Proper CDN Implementation
I am working with a site that is using the Cloudflare CDN to deliver their images. The problem is that the URLs for the images are all cloudflare URLs (cloudflare.com/example-filename.png). We want to use a cname to change the image URLs to be on a subdomain of their site to take advantage of the SEO strength. Currently, they get a high percentage of traffic from image search and Google is directing users to their site when returning images hosted on Cloudflare, so I am apprehensive about making changes. I'm curious if anyone has any experience with this or has any insight on how Google will handle it. Will the longer term benefits out way any short term drop in traffic from image search? WIll there be a drop in image search? Any thoughts or experiences are appreciated. Thanks in advance.
Intermediate & Advanced SEO | | highlyrelevant1 -
Best way to re-order page elements based on search engine users
Both versions of the page has essentially same content, but in different order. One is for users coming from Google (and google bot) and other is for everybody else. Questions: Is it cloaking? what will be the best way to re-order elements on the page: totally different style sheets for each version, or calling in different divs in a same style sheet? Is there any better way to re-order elements based on search engine? Let me make it clear again: the content is same for everyone, just in different order for visitors coming from Google and everybody else. Don't ask me the reason behind it (executive orders!!)
Intermediate & Advanced SEO | | StickyRiceSEO0 -
How to retain link juice moving to new site, cms and servers?
We have been hosting our website with a provider (their design and CMS) and we are now moving to a new design, better content focussing on keywords in a different CMS platform on different servers but want to retain the link juice from the old site. We have used Open Site Explorer Report to determine all the links to the old site and the pages they link to. What is the best strategy to keep the link juice flowing to the new site? Example This site <http: www.dogslifedownunder.com="" what-is-worse-then-going-to-the-v-e-t="">links to this page <http: 19105="" www.sydneyanimalhospitals.com.au="" ourstaff="" thevets="" tabid="" default.aspx="">on the old site.</http:></http:> We will have a similar page on the new site with the same staff members called for example: How do we ensure that the we retain the link juice? Any thoughts most welcome.
Intermediate & Advanced SEO | | Peter.Huxley590