Moving Code for Faster Crawl Through?
-
What are best practices for moving code into other folders to help speed up a crawling for bots? We once moved some javascript from an SEO's suggestion and the site suddenly looked like crap until we undid the changes. How do you figure our what code should be consolidated? What code do you use to indicate what has been moved and to where?
-
Yes, I misread your post.
Best,
Christopher -
But isn't this really for things like Google Analytics code? The code I am talking about is stuff like CSS and Javascript that could be in external files but are not currently.
-
But isn't this really for things like Google Analytics code? The code I am talking about is stuff like CSS and Javascript that could be in external files but are not currently.
-
Have you considered using Google Tag Manager? It's much cleaner, and the scripts run asynchronously to speed up page load times.
Best,
Christopher
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Will google be able to crawl all of the pages given that the pages displayed or the info on a page varies according to the city of a user?
So the website I am working for asks for a location before displaying the product pages. There are two cities with multiple warehouses. Based on the users' location, the product pages available in the warehouse serving only in that area are shown. If the user skips location, default warehouse-related product pages are shown. The APIs are all location-based.
Intermediate & Advanced SEO | | Airlift0 -
Crawl and Indexation Error - Googlebot can't/doesn't access specific folders on microsites
Hi, My first time posting here, I am just looking for some feedback on a indexation issue we have with a client and any feedback on possible next steps or items I may have overlooked. To give some background, our client operates a website for the core band and a also a number of microsites based on specific business units, so you have corewebsite.com along with bu1.corewebsite.com, bu2.corewebsite.com. The content structure isn't ideal, as each microsite follows a structure of bu1.corewebsite.com/bu1/home.aspx, bu2.corewebsite.com/bu2/home.aspx and so on. In addition to this each microsite has duplicate folders from the other microsites so bu1.corewebsite.com has indexable folders bu1.corewebsite.com/bu1/home.aspx but also bu1.corewebsite.com/bu2/home.aspx the same with bu2.corewebsite.com has bu2.corewebsite.com/bu2/home.aspx but also bu2.corewebsite.com/bu1/home.aspx. Therre are 5 different business units so you have this duplicate content scenario for all microsites. This situation is being addressed in the medium term development roadmap and will be rectified in the next iteration of the site but that is still a ways out. The issue
Intermediate & Advanced SEO | | ImpericMedia
About 6 weeks ago we noticed a drop off in search rankings for two of our microsites (bu1.corewebsite.com and bu2.corewebsite.com) over a period of 2-3 weeks pretty much all our terms dropped out of the rankings and search visibility dropped to essentially 0. I can see that pages from the websites are still indexed but oddly it is the duplicate content pages so (bu1.corewebsite.com/bu3/home.aspx or (bu1.corewebsite.com/bu4/home.aspx is still indexed, similiarly on the bu2.corewebsite microsite bu2.corewebsite.com/bu3/home.aspx and bu4.corewebsite.com/bu3/home.aspx are indexed but no pages from the BU1 or BU2 content directories seem to be indexed under their own microsites. Logging into webmaster tools I can see there is a "Google couldn't crawl your site because we were unable to access your site's robots.txt file." This was a bit odd as there was no robots.txt in the root directory but I got some weird results when I checked the BU1/BU2 microsites in technicalseo.com robots text tool. Also due to the fact that there is a redirect from bu1.corewebsite.com/ to bu1.corewebsite.com/bu4.aspx I thought maybe there could be something there so consequently we removed the redirect and added a basic robots to the root directory for both microsites. After this we saw a small pickup in site visibility, a few terms pop into our Moz campaign rankings but drop out again pretty quickly. Also the error message in GSC persisted. Steps taken so far after that In Google Search Console, I confirmed there are no manual actions against the microsites. Confirmed there is no instances of noindex on any of the pages for BU1/BU2 A number of the main links from the root domain to microsite BU1/BU2 have a rel="noopener noreferrer" attribute but we looked into this and found it has no impact on indexation Looking into this issue we saw some people had similar issues when using Cloudflare but our client doesn't use this service Using a response redirect header tool checker, we noticed a timeout when trying to mimic googlebot accessing the site Following on from point 5 we got a hold of a week of server logs from the client and I can see Googlebot successfully pinging the site and not getting 500 response codes from the server...but couldn't see any instance of it trying to index microsite BU1/BU2 content So it seems to me that the issue could be something server side but I'm at a bit of a loss of next steps to take. Any advice at all is much appreciated!0 -
Best way to link to 1000 city landing pages from index page in a way that google follows/crawls these links (without building country pages)?
Currently we have direct links to the top 100 country and city landing pages on our index page of the root domain.
Intermediate & Advanced SEO | | lcourse
I would like to add in the index page for each country a link "more cities" which then loads dynamically (without reloading the page and without redirecting to another page) a list with links to all cities in this country.
I do not want to dillute "link juice" to my top 100 country and city landing pages on the index page.
I would still like google to be able to crawl and follow these links to cities that I load dynamically later. In this particular case typical site hiearchy of country pages with links to all cities is not an option. Any recommendations on how best to implement?0 -
Would this be considered cloaking and would it be a bad move?
I posted this topic last night, http://moz.com/community/q/seo-dealing-with-a-cdn-on-a-site about issues I am having with a client's images falling out of index because they have a CDN now. So I have come up with a work around, but it might be considered cloaking and I am not sure. A month ago we changed over to using a CDN and the images started falling out of the index after that. Currently when you land on a page the images are served from cdn.site.com What I am thinking about doing is detecting Google Bot and when Google Bot crawls the site serve images from site.com. The images will be the exact same images as served from the CDN so it is not a content switcharoo thing. It is just to try to get the images back in the index. So would this be considered cloaking in your opinion?
Intermediate & Advanced SEO | | LesleyPaone0 -
How complex or what to consider when moving from a .aspx webdeveloper to my own wordpress.org website?
Basically my current web developer is not providing me with what a modern website should need to fully utilize online marketing and SEO in terms of blogging, social media widgets, e-commerce and so on. Because of this I have thought of moving to a wordpress.org website run and built by myself. Is this a good idea? What is the best way to migrate and save existing authority (Re-directs etc)? Is there any potential risks or problems that I could encounter that aren't immediate obvious? Many thanks! Tom
Intermediate & Advanced SEO | | CoGri0 -
Moving from Shopp to WooCommerce...
My site is ranking well on most of my products and honestly, I don't do the best job in "on product page" SEO. I've got solid URLs for each, but my "on page" grades are mostly C, D and Fs. 😞 I honestly feel fortunate that I'm ranking well and have "put off" doing much work knowing that I was going to change from the Shopp wordpress plugin to Woocommerce. Is there anything structurally that will potentially hurt me from changing framework? Do I need to make sure that the URLs are the EXACT same for each of my products or will it be okay to make them better? Ex. some pages have a less desirable URL because I just "copied" the previous item. Thanks in advance for your experience.
Intermediate & Advanced SEO | | dp428280 -
Old pages still crawled by SE returning 404s. Better to put 301 or block with robots.txt ?
Hello guys, A client of ours has thousand of pages returning 404 visibile on googl webmaster tools. These are all old pages which don't exist anymore but Google keeps on detecting them. These pages belong to sections of the site which don't exist anymore. They are not linked externally and didn't provide much value even when they existed What do u suggest us to do: (a) do nothing (b) redirect all these URL/folders to the homepage through a 301 (c) block these pages through the robots.txt. Are we inappropriately using part of the crawling budget set by Search Engines by not doing anything ? thx
Intermediate & Advanced SEO | | H-FARM0 -
How will moving my website off of DNN platform affect my SEO?
Website is currently in Dot Net Nuke (DNN) and planning on moving it into a different platform, possibly Sitecore. How will shifting CMS affect SEO efforts? Thank you
Intermediate & Advanced SEO | | Unidev0