Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Forced Redirects/HTTP<>HTTPS 301 Question
Hi All, Sorry for what's about to be a long-ish question, but tl;dr: Has anyone else had experience with a 301 redirect at the server level between HTTP and HTTPS versions of a site in order to maintain accurate social media share counts? This is new to me and I'm wondering how common it is. I'm having issues with this forced redirect between HTTP/HTTPS as outlined below and am struggling to find any information that will help me to troubleshoot this or better understand the situation. If anyone has any recommendations for things to try or sources to read up on, I'd appreciate it. I'm especially concerned about any issues that this may be causing at the SEO level and the known-unknowns. A magazine I work for recently relaunched after switching platforms from Atavist to Newspack (which is run via WordPress). Since then, we've been having some issues with 301s, but they relate to new stories that are native to our new platform/CMS and have had zero URL changes. We've always used HTTPS. Basically, the preview for any post we make linking to the new site, including these new (non-migrated pages) on Facebook previews as a 301 in the title and with no image. This also overrides the social media metadata we set through Yoast Premium. I ran some of the links through the Facebook debugger and it appears that Facebook is reading these links to our site (using https) as redirects to http that then redirect to https. I was told by our tech support person on Newspack's team that this is intentional, so that Facebook will maintain accurate share counts versus separate share counts for http/https, however this forced redirect seems to be failing if we can't post our links with any metadata. (The only way to reliably fix is by adding a query parameter to each URL which, obviously, still gives us inaccurate share counts.) This is the first time I've encountered this intentional redirect thing and I've asked a few times for more information about how it's set up just for my own edification, but all I can get is that it’s something managed at the server level and is designed to prevent separate share counts for HTTP and HTTPS. Has anyone encountered this method before, and can anyone either explain it to me or point me in the direction of a resource where I can learn more about how it's configured as well as the pros and cons? I'm especially concerned about our SEO with this and how this may impact the way search engines read our site. So far, nothing's come up on scans, but I'd like to stay one step ahead of this. Thanks in advance!
Technical SEO | | ogiovetti0 -
Can I redirect a link even if the link is still on the site
Hi Folks, I've got a client who has a duplicate content because they actually create duplicate content and store the same piece of content in 2 different places. When they generate this duplicate content, it creates a 2nd link on the site going to the duplicate content. Now they want the 2nd link to always redirect to the first link, but for architecture reasons, they can't remove the 2nd link from the site navigation. We can't use rel-canonical because they don't want visitors going to that 2nd page. Here is my question: Are there any adverse SEO implications to maintaining a link on a site that always redirects to a different page? I've already gone down the road of "don't deliberately create duplicate content" with the client. They've heard me, but won't change. So, what are your thoughts? Thanks!
Technical SEO | | Rock330 -
Can you have an SSL cert but still have http?
I was under the impression that if you got an SSL cert for your site that the site would change to https. I ran this site: http://thekinigroup.com/ through an SSL checker and it said it had one...but it's http. 1. Why didn't it change to https? Is there an extra step there that needs to be done? 2. Is there a reason someone would choose to get an SSL cert, but not have https? Thanks, Ruben
Technical SEO | | KempRugeLawGroup0 -
Rel="canonical" What if there is no header??
Hi Everyone! Thanks to moz.com, I just found out that we have a duplicate content issue: mywebsite.com and mywebsite.com/index.php have the same content. I would like to make mywebsite.com the main one because it already has a few links and a better page rank. I know how to do a 301 redirect (already have one for www.mywebsite.com) but I am aware that a 301 redirect for my index file would create a loop issue. I have read the article about redirecting without creating a loop (http://moz.com/blog/apache-redirect-an-index-file-to-your-domain-without-looping) but quite frankly I don't even have a clue what he's trying to tell me (e.g. "Create an apache DirectoryIndex directive for your document root." What????!)… So I figured a rel="canonical" tag for my index file would be easier and fix the problem, too (right??) In every "How to" description they always say you have to put the rel="canonical" tag in the header of your duplicate content file. But: My index.php has no header (or nothing that looks like a header to me)! This is what it looks like: foreach($_GET as $key => $value)
Technical SEO | | momof4
{
$$key = $value;
}
foreach($_POST as $key => $value)
{
$$key = $value;
}
$page_title="my title";
$page_description="my description";
$page_keywords="keywords";
//echo $link;
//exit;
if (!isset($link)):
$page_content="homepage.php";
else:
if ($link=="services"):
$page_content="services.php";
$page_title=" my title for services page";
$page_description="description for services.";
endif;
… ect. for the other pages So where do I put the rel=canonical tag? Or is there another solution for the whole problem? Like delete the whole index file (lol) Thanks in advance for any answers!0 -
Expired Domain - http:// or www
I have an old domain - When i use the link explorer i get way more juice out of the www version of my domain. I will be using wordpress to set up a new domain with the same name . My question is - How do I make it proper for seo? Do i just change the http:// to www in wordpress and be done with it? Does it even matter (thinking it does)
Technical SEO | | imagatto20 -
Page authority has disappeared since redirect
As the title states, my page authority (PA) has completely disappeared for certain pages on my site.
Technical SEO | | Hughescov
I have recently had a redesign and optimised the page names, redirecting the page it has replaced.
This has been nearly 2 months now and the new pages are showing PA of 1 still.
The 301 redirects definitely work so what am I doing wrong? Thanks in advance!0 -
Non-Canonical Pages still Indexed. Is this normal?
I have a website that contains some products and the old structure of the URL's was definitely not optimal for SEO purposes. So I created new SEO friendly URL's on my site and decided that I would use the canonical tags to transfer all the weight of the old URL's to the New URL's and ensure that the old ones would not show up in the SERP's. Problem is this has not quite worked. I implemented the canonical tags about a month ago but I am still seeing the old URL's indexed in Google and I am noticing that the cache date of these pages was only about a week ago. This leads me to believe that the spiders have been to the pages and seen the new canonical tags but are not following them. Is this normal behavior and if so, can somebody explain to me why? I know I could have just 301 redirected these old URL's to the new ones but the process I would need to go through to have that done is much more of a battle than to just add the canonical tags and I felt that the canonical tags would have done the job. Needless to say the client is not too happy right now and insists that I should have just used the 301's. In this case the client appears to be correct but I do not quite understand why my canonical tags did not work. Examples Below- Old Pages: www.awebsite.com/something/something/productid.3254235 New Pages: www.awebsite.com/something/something/keyword-rich-product-name Canonical tag on both pages: rel="canonical" href="http://www.awebsite.com/something/something/keyword-rich-product-name"/> Thanks guys for the help on this.
Technical SEO | | DRSearchEngOpt0 -
New website branding, differences between http://www and http://
Hey Mozers! We will be creating another brand pretty soon with some pretty cool interactive features and before we start development of the site I was wondering if there are any pros/cons to branding the site sans the www. For example http://example123.com and http://www.example123.com. I would much prefer to brand it has http://example123.com but I just wanted to check first to see if that would have any negative SEO ramifications. It seems that it might just be a preference as I looked at Facebook and Twitter and they both do it differently, same with Groupon and LivingSocial. Looking forward to hearing from you guys!
Technical SEO | | Riggz1