Bots gone from site for 1-2 hours
-
Hi everyone,
From the beginning of March bots and crawlers begin to disappear from site completely for 1-2 hours and then come back again. It never happened earlier, they constantly crawled our site without stopping. We didn't make any changes on the site.
After that pages and images start to disappear from G index.
Does anyone have the same trouble?
-
Assuming you are talking about the website in your profile...here is what I would check: Do you have any content duplication between your multiple language subdomains ? Do you have XML Sitemaps for your pages ? Are they submitted via the Webmaster Console ? If yes, what %age of your pages do you see indexed vs how many you submitted ? Have you considered doing an Images XML Sitemap ?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Re-using site code.
Hi, I'm looking at launching a new website, and am keen to understand whether re-using the basic code behind one of my other sites will cause me an issue. I'll be changing the directory structure/ file names, etc - but it will basically leave me with a very similar-looking site to another in my portfolio - using code thats all ready out there, etc. Thanks, David
Technical SEO | | newstd1000 -
Configure a mobile site with WMT
Hello Everyone, I'm in a situation that I have no idea how to handle. I have only really dealt with RWD, and not a mobile-specific site. Anyway, I have a client who is launching an m.domian.com for their mobile site, how do I add/configure this in WMT? Thanks Zach
Technical SEO | | Zachary_Russell0 -
Why is this site ranking better than me
Hi just used the compare tool to try and find out why a site is ranking better than me http://www.opensiteexplorer.org/comparisons?site=www.lifestylemonthly.co.uk%2F my site is www.in2town.co.uk and the site i am comparing with is http://www.lifestylemonthly.co.uk/ Can anyone explain what is going on and how i can achieve better ranking results
Technical SEO | | ClaireH-1848860 -
What can we do to improve our site
Hi. I am hoping that some of you can help me with the in2town site www.in2town.co.uk The site is a news/lifestyle magazine site. The site is a cross between, huffington post, digital spy, female first and the sun newspaper. Basically the site is a news site as well as covering showbiz news, travel news, health news and advice etc What i would like is for people to look at the site and let me know what they feel i should do to improve the site to make it better for our readers and to gain more readership. I would also like to hear from people on how they find moving around the site as well as the speed of the site. At the moment the site is with an american hosting company and i am in the process of talking to UK hosting companies to move the site. The site is currently on a dedicated server. It would mean a lot if people could give me their advice on how to improve the site and make it a beter experience for our readers while at the same time being able to generate income with the site. Just a quick note, all content is original and we have a number of people who write for the site. many thanks
Technical SEO | | ClaireH-1848860 -
How to move a site slowly
I've got a site that I started back in 2001 (www.jaaroncaststone.com) to sell what I was making, concrete countertops and sinks. Well we're going to be discontinuing the concrete products in a month or so and want to start pointing all the links from the old site to a new one dedicated to a single product (www.jaaron-wood-countertops.com). I don't want to do a full 301 on the old site just yet but can I put a canonical tag on the index and about us pages pointing to the new site now so the search engines start looking that way or should I wait and do a full 301 when the day comes to pull the old site?
Technical SEO | | JAARON0 -
About Bot's IP
Hi, one of my customers had probably block the IP of SEOMOZ's bot. Could you give me : IP User-agent's name thks for helping me 😉
Technical SEO | | dawa1 -
Are lots of links from an external site to non-existant pages on my site harmful?
Google Webmaster Tools is reporting a heck of a lot of 404s which are due to an external site linking incorrectly to my site. The site itself has scraped content from elsewhere and has created 100's of malformed URLs. Since it unlikely I will have any joy having these linked removed by the creator of the site, I'd like to know how much damage this could be doing, and if so, is there is anything I can do to minimise the impact? Thanks!
Technical SEO | | Nobody15569050351140 -
Site Structure question
when deciding the Site structure for a e-commerce site Is it better to keep everything mysite.com/widget.html or use categories like mysite.com/Gifts/widget.html
Technical SEO | | DavidKonigsberg0