404s in Google Search Console and javascript
-
The end of April, we made the switch from http to https and I was prepared for a surge in crawl errors while Google sorted out our site. However, I wasn't prepared for the surge in impossibly incorrect URLs and partial URLs that I've seen since then.
I have learned that as Googlebot grows up, he'she's now attempting to read more javascript and will occasionally try to parse out and "read" a URL in a string of javascript code where no URL is actually present. So, I've "marked as fixed" hundreds of bits like
/TRo39,
category/cig
etc., etc....But they are also returning hundreds of otherwise correct URLs with a .html extension when our CMS system generates URLs with a .uts extension like this:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.html
when it should be:
https://www.thompsoncigar.com/thumbnail/CIGARS/90-RATED-CIGARS/FULL-CIGARS/9012/c/9007/pc/8335.utsWorst of all, when I look at them in GSC and check the "linked from" tab it shows they are linked from themselves, so I can't backtrack and find a common source of the error.
Is anyone else experiencing this? Got any suggestions on how to stop it from happening in the future? Last month it was 50 URLs, this month 150, so I can't keep creating redirects and hoping it goes away.
Thanks for any and all suggestions!
Liz Micik -
Hi Liz,
What I would do as well is go with a solution around your robots.txt to make sure the crawlers will respect it and don't go on a hunch trying to find new URLs that are embedded somewhere else. Usually it's something you shouldn't worry about too much, it's just the crawler doing a good job trying to find more content/URLs on your site.
Martijn.
-
Google Search Console 404s can be very irritating.
What does your robots.txt file currently say?
You can remove all instances of .html in your htaccess file to help solve for this issue. More on that here: https://alexcican.com/post/how-to-remove-php-html-htm-extensions-with-htaccess/
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Search Console Click Through Results
Hi Moz Community, I've newly started using Google's search console on a regular basis, and I have a query about the results coming from the platform. For example, I added this article URL to the console with field "URL containing" and "URL exact", and the results showed 71 clicks and 436 impressions over 12 months. However, when looking at the queries that show in the results to see where the clicks are coming from, all of them state there's no clicks from the dozen or so queries that show. Am I doing something wrong here? Is there a field I need to fill in to show more results/top results? I've tested a few article URL's and the same has happened. Thanks in advance for any help.
Algorithm Updates | | FunktionEvents1 -
When Google is going to penalise webpages with autoplaying videos?
Hi, One of the most disgusting experiences while browsing web pages is the auto playing anonymous and highly non relevant ad videos at some corner of the page which buffer mostly and automatically plays. There are familiar websites which been following this activity. I don't think Google has ever rolled out an update to penalise these. Any idea why? Thanks
Algorithm Updates | | vtmoz0 -
What Ranking Factors Impact Google News Visibility?
I'm just at the beginning of a new analysis involving Google News visibility and ranking factors, and thought I could put the project out to you, dear SEO geniuses, to get your ideas and perspectives. Backgrounder: My company operates over 50 niche, disease-specific daily news sites, covering science, research and advocacy news about specific diseases. Virtually all of them are in Google News. They range in age from 3 years old to 3 months old. Varying degrees of page rank / authority Content on the site is completely niche to specific diseases, and we have a lot of sites for rare and orphan diseases. Most of the content is news, but we also have info/resources pages, blogs, and some short-form posts made for use in social media. The Project: I want to do an analysis of keywords in our news headlines and see how certain keywords correlate with articles that do well -- both in terms of search traffic and overall with users. Going to use our Multiple Sclerosis News Today website. Most of our search traffic comes from Google News. What I hope to gain: I'm curious to see if certain sets of keywords that relate to the disease, to therapies, etc. drive the most traffic. I want to compare these keyword lists to how well we rank in organic search for the same keywords (via news articles or info pages) to see if there is a connection. I want to also create a working keyword list of the best-performing keywords in the news as a way of cross-pollinating content production on our blogs, info pages, social content, etc. I want to increase my knowledge base of ranking factors specific for Google News. The last point is really something I wish I knew more about. I feel like there aren't many knowledge resources out there about Google News. Is it safe to assume that the same on-site and off-site SEO best practices that govern organic search engine visibility are at play in Google News, or are there independent factors as well? I'd love to get your thoughts. Thanks!
Algorithm Updates | | Michael_Nace0 -
Google is really NOT SAYING IN "HOW SEARCH WORKS” ?
Hi All SEOmoz members and team, As I was reading this, is it true that Google does this . Simply, I don't think so, I haven't experienced any of such what is being talked [http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/ C](http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/ "http://www.fairsearch.org/search-manipulation/what-google-isnt-saying-in-how-search-works/")ome on, let us discuss the real thing about Google. Teginder Ravi
Algorithm Updates | | Futura0 -
How to show your ratings in the Google SERP
I've noticed that some organic search results are showing ratings just above the meta tag. How are these sites doing this? Example: If you search "cash advance", there is a result between #4 and #6 in the organic results. The site is "goldcashadvance.com". It's showing a 5-star rating in the result.
Algorithm Updates | | sparagi0 -
New Search Engine.... Vanoogle.com
I'd like to see google start a new search engine. They might call it Vanoogle.com (Vanilla Google). This search engine would not be stinked up with social data, freshness inclusions, crap from my last query, skewed based upon my IP, warped because of my browser, targeted because of my cookies, no personalization, no image results, product results, none of that stuff. Ads are OK if labeled. I just want a plain vanilla search. Something that I know is "clean". Just like the good olde days. Millions of people will start using it right away. 🙂 Would you use Vanoogle.com?
Algorithm Updates | | EGOL2 -
Source for how many searches are done on Google per day?
Hi, All! The figure I've seen going around is 3 billion and is attributed to ComScore, but in the comScore press release that was linked to (actually the one from Rand's article - http://www.seomoz.org/blog/21-tactics-to-increase-blog-traffic-2012) all I could find was percentage of market share, but no total numbers of searches. Anyone have a source on that? Thanks, Aviva
Algorithm Updates | | debi_zyx0 -
Why is a website with lower content interest reaching higher in google
there is a website that i am competing with <cite>www.gastricbandhypnotherapy.net for the term gastric band hypnotherapy and for some reason it is now ranching higher than me.</cite> I have been number one in google with http://www.clairehegarty.co.uk/virtual-gastric-band-with-hypnotherapy for the term Gastric Band Hypnotherapy but for some reason in the past few days it has ranked number one and pushed me down to number three. i do not understand it as there is not much relevant content to gastric band hypnotherapy and also it does not have many links pointing into it can you please help with this question
Algorithm Updates | | ClaireH-1848860