Is the If-Modified-Since HTTP Header still relevant?
-
I'm relatively new to the technical side of SEO and have been trying to brush up my skills by going through Google's online Web-master Academy, which suggests that you need a If-Modified-Since HTTP Header tag on your site. I checked and apparently our web server doesn't support this.
I've been told by a good colleague that the If-Modified-Since tag is no longer relevant as the spiders will frequently revisit a site as long as you regularly update and refresh the content (which we do).
However our site doesn't seem to of been reindexed for a while as the cached version's are still showing the pages from over a month ago.
So two question really - is the If-Modified-Since HTTP Header still relevant and should I make sure this is included?
And is there anything else I should be doing to make sure the spiders crawl our pages? (apart from keeping them nice, fresh and useful)
-
If the webserver does not support (or the admin does not want to enable) this feature you could always have your frontend-templates have a small string wich holds the date/time when the page was last updated. Something along the lines "last updated on: ...." at the bottom or top of the content area. It's also an useful bit of information for users.
-
Hi Annie
I'm surprised there hasn't been lots of answers to your question.
Check-out this video here on SEOmoz entitled "Whiteboard Interview - Google's Matt Cutts on Redirects, Trust + More" featuring Matt Cutts being asked some questions by Rand. It opens with a partial answer to your first question:
"These days we use it a little less" (2 years ago) ~ basically means that in locations such as the US, most of Europe, Japan... & so on, where Bandwidth is rarely an issue anymore, 'If-Modified-Since' isn't taken notice of, it's not worth including anymore.
In say developing countries where bandwidth is sometimes still on the low side, it may still be used, hence why a sweeping 'it doesn't matter anymore' statement wasn't given.
**Your second question: **
- Content, fresh unique value-adding content that is, that's engaging and shareable, is always a positive aspect to work on, which in turn can lead to some awesome new links. This encourages the bots to visit more regularly.
- Ensuring that your site doesn't have any technical issues (say causing significant downtime).
- Ensuring that Robots.txt isn't wrongly disallowing any pages from being crawled.
- Keeping an eye on Google Webmaster Tools (& Bing Webmaster Tools) for any messages or errors.
- You can alter the crawl rate in GWT, though is usually best to leave it on the default auto setting.
Hope that helps,
Simon
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Http v https Duplicate Issues
Hello, I noticed earlier an issue on my site. http://mysite.com and https://mysite.com both had canonical links pointing to themselves so in effect creating duplicate content. I have now taken steps to ensure the https version has a canonical that points to the http version but I was wondering what other steps would people recommend? Is it safe to NOINDEX the https pages? Or block them via robots.txt or both? We are not quite ready to go fully HTTPS with our site yet (I know Google now prefers this) Any thoughts would be very much appreciated.
Technical SEO | | niallfred0 -
My old URL's are still indexing when I have redirected all of them, why is this happening?
I have built a new website and have redirected all my old URL's to their new ones but for some reason Google is still indexing the old URL's. Also, the page authority for all of my pages has dropped to 1 (apart from the homepage) but before they were between 12 to 15. Can anyone help me with this?
Technical SEO | | One2OneDigital0 -
Robots.txt on http vs. https
We recently changed our domain from http to https. When a user enters any URL on http, there is an global 301 redirect to the same page on https. I cannot find instructions about what to do with robots.txt. Now that https is the canonical version, should I block the http-Version with robots.txt? Strangely, I cannot find a single ressource about this...
Technical SEO | | zeepartner0 -
Is "commented out" text still read by the SEs?
A site I reviewed was showing up in Google rankings for key phrases specific to a city, however the page that was showing up had the 'city' key phrases commented out. Does Google still read and utilized commented out text? Or is it more likely that the page in question got indexed before the key phrases were commented out and it's just still appearing for the related search queries?
Technical SEO | | MLTGroup1 -
Site-Wide Header image w/ Link hurting me?
I have a banner in the header that is constant across all pages except the one it links to. It goes in before the content div and this search term is #1 across google in almost every variation, but it appears to link in a lot of the non relevant pages for #2 spots in some cases. While this is a relatively new domain (started 03/12), but this has been a constant rank for about 2 months. I'm wondering if this may be hurting the keyword targeting of my internal pages and if i should no follow that header image on all pages except the homepage?
Technical SEO | | choiceenergy0 -
Multiple h1 headers in a slideshow
I’ve built a slideshow and each slide contains an image, h1 and text contained within a div. The slideshow grabs the whole div and slides it. My question is should I remove the h1 and style it with css e.g. my slide header or leave in the multiple h1 headers. All thoughts and comments welcome.
Technical SEO | | RodneyRiley0 -
Is having a sitemap.xml file still beneficial?
Hi, I'm pretty new to SEO and something I've noticed is that a lot of things become relevant and irrelevant like the weather. I was just wondering if having a sitemap.xml file for Google's use is still a good idea and beneficial? Logically thinking, my websites would get crawled faster by having one. Cheers.
Technical SEO | | davieshussein0 -
Does domain registrar location (still?) play a role in SEO
I was wondering if the location of our domain registrar still play a role in SEO. The thing is that one of our domains are registered with a registrar in another country. It is a national TLD (.no). Could this have any influence on our rankings? The reason for asking is that i came across this article at Search Engine Journal:
Technical SEO | | AndersS
http://www.searchenginejournal.com/google-rankings-influenced-by-server-domain-registrar-and-geographic-location/5963/ saying: "
Hosting is not the only variable Google may look at when determining search relevance for a site, here is a look at hosting, domain registrar information and geographic location." I was wondering if what is said here, still is valid (article is from 2007). Hope you guys can help me out.0