How can I exclude display ads from robots.txt?
-
Google has stated that you can do this to get spiders to content only, and faster. Our IT guy is saying it's impossible.
Do you know how to exlude display ads from robots.txt?Any help would be much appreciated.
-
You'd want to make the URL paths where the display ads live to have the crawl disallowed in your robots.txt, just like any other section of your site. Here's some basics on robots.txt.
Hope this helps!
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
2 sitemaps on my robots.txt?
Hi, I thought that I just could link one sitemap from my site's robots.txt but... I may be wrong. So, I need to confirm if this kind of implementation is right or wrong: robots.txt for Magento Community and Enterprise ...
Technical SEO | | Webicultors
Sitemap: http://www.mysite.es/media/sitemap/es.xml
Sitemap: http://www.mysite.pt/media/sitemap/pt.xml Thanks in advance,0 -
Wordpress BackupBuddy adding ?doing_wp_cron= in URLS
Hi Has anyone found WordPress Backup Buddy causing a problem with SEO. I understand why it does it, but wondered if anyone experienced issues with this? Only sometimes it adds /?doing_wp_cron=****** on to the end of a URL Thanks Tom
Technical SEO | | TomPryor831 -
Robots.txt Download vs Cache
We made an update to the Robots.txt file this morning after the initial download of the robots.txt file. I then submitted the page through Fetch as Google bot to get the changes in asap. The cache time stamp on the page now shows Sep 27, 2013 15:35:28 GMT. I believe that would put the cache time stamp at about 6 hours ago. However the Blocked URLs tab in Google WMT shows the robots.txt last downloaded at 14 hours ago - and therefore it's showing the old file. This leads me to believe for the Robots.txt the cache date and the download time are independent. Is there anyway to get Google to recognize the new file other than waiting this out??
Technical SEO | | Rich_A0 -
CSS Float Top Left Image not displaying
Using CSS/HTML in Dreamweaver CC, I am trying to display a SEO friendly matrix of six images wide by three deep on the Home Page, but cannot get the first (top left) image to display consistently. The page appears correctly in Dreamweaver Design View, but not in the Live View. Equally it works fine in Safari, but not in Firefox nor I.E. I seem to have tried every CSS variation to resolve this issue, without success. Can someone stear me in the right direction, please? The relevant HTML Code ... Celotex PL4050 - 62.5mm insulated wallboard from £11.96 per m² / £34.45 per board SuperFoil SF19 40mm Multi-foil Roll from £5.23 per m² / £98.00 per 18.75 m² Roll Celotex GA4000 General application Rigid PIR from £8.56 per m² / £24.66 per board Rockwool Full-fill 75mm Cavity Wall Slabs from £3.51 per m² / £1.92 per slab Knauf Earthwool 270mm Loft Roll 44 Combi-Cut from £3.80 per m² / £22.51 per 9.93 m² Roll Kingspan Kooltherm K8 75mm Cavity Wall Boards from £16.26 per m² / £8.79 per board Knauf Earthwool 170mm Combi-cut Loft Roll 44 from £3.16 per m² / £25.34 per 8 m² Roll Kingspan Kooltherm K7 Rigid Phenolic Foam - 75mm £13.91 per m² / £40.05 per board ... The associated CSS code ... #popular {
Technical SEO | | JustInsulation
width: 1050px;
height: 800px;
overflow: hidden;
width: auto;
height: auto;
}
#product1 {
background-image: url(../007-graphics/popular/01-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 24px;
float: left;
overflow: auto;
clear: left;
}
#product2 {
background-image: url(../007-graphics/popular/02-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
opacity: 0.85;
}
#product3 {
background-image: url(../007-graphics/popular/03-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
opacity: 0.85;
}
#product4 {
background-image: url(../007-graphics/popular/04-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
}
#product5 {
background-image: url(../007-graphics/popular/05-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
opacity: 0.85;
}
#product6 {
background-image: url(../007-graphics/popular/06-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
opacity: 0.85;
}
#product7 {
clear: both;
background-image: url(../007-graphics/popular/07-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 24px;
float: left;
max-height: 275px;
opacity: 0.85;
}
#product8 {
background-image: url(../007-graphics/popular/08-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
}
#product9 {
background-image: url(../007-graphics/popular/09-prod.png);
background-repeat: no-repeat;
width: 150px;
height: 265px;
margin-left: 10px;
float: left;
max-height: 275px;
opacity: 0.85;
} ... The complete code is located at - www.just-insulation.com/index.html0 -
SEO impact of adding a forum module to a site
I help maintain a site and the owners want to start a forum for their club members. This is a site that runs on Joomla, so it's likely we'll use some form of a Joomla module to add the forum functionality. My main question is whether there is anything to be concerned about for SEO purposes when adding the forum. I've never set one up, so I don't know the best practices for forums/discussion boards (although I'm scouring the web for general info). Should it be a subdomain or on the main domain -- or does this not matter? Are there any general setup processes that I should pay attention to for SEO best-practices? This question was mainly about the SEO impact, but if anyone can point me in the direction of forum best practice info as well, It would be appreciated. Cheers!
Technical SEO | | kariel
Kristy0 -
How can something be duplicate content of itself?
Just got the new crawl report, and I have a recurring issue that comes back around every month or so, which is that a bunch of pages are reported as duplicate content for themselves. Literally the same URL: http://awesomewidgetworld.com/promotions.shtml is reporting that http://awesomewidgetworld.com/promotions.shtml is both a duplicate title, and duplicate content. Well, I would hope so! It's the same URL! Is this a crawl error? Is it a site error? Has anyone seen this before? Do I need to give more information? P.S. awesomewidgetworld is not the actual site name.
Technical SEO | | BetAmerica0 -
Robots.txt question
What is this robots.txt telling the search engines? User-agent: * Disallow: /stats/
Technical SEO | | DenverKelly0 -
Is it terrible to not have robots.txt ?
I was under the impression that you really should have a robots.txt page, and not having one is pretty bad. However, hubspot (which I'm not impressed with) does not have the capability of properly implementing one. Will this hurt the site?
Technical SEO | | StandUpCubicles1