Crawl questions
-
My first website crawl indicating many issues. I corrected the issues, requested another crawl and received the results. After viewing the excel file I have some questions.
1. There are many pages with missing Titles and Meta Descriptions in the Excel file. An example is http://www.terapvp.com/threads/help-us-decide-on-terapvp-com-logo.25/page-2
That page clearly has a meta description and title. It is a forum thread. My forum software does a solid job of always providing those tags. Why would my crawl report not show this information? This occurs on numerous pages.
2. I believe all my canonical URLs are properly set. My crawl report has 3k+ records, largely due to there being 10 records for many pages. These extra records are various sort orders and style differences for the same page i.e. ?direction=asc.
My need for a crawl report is to provide actionable data so I can easily make SEO improvements to my site where necessary. These extra records don't provide any benefit.
IF the crawl report determined there was not a clear canonical URL, then I could understand. But that is not the case. An example is http://www.terapvp.com/forums/news/ If you look at the source you will clearly see
Where is the benefit to including the 10 other records in the Crawl report which show this same page in various sort orders? Am I missing anything?
3. My robots.txt appropriately blocks many pages that I do not wish to be crawled. What is the benefit to including these many pages in the crawl report?
Perhaps I am over analyzing this report. I have read many articles on SEO, but now that I have found SEOmoz, I can see I will need to "unlearn what I have learned". Many things such as setting meta keyword tags are clearly not helpful. I wish to focus my energy and I was looking to the crawl report as my starting point. Either I am missing something, or the report design needs improvement.
-
Those are commends added to the code. My site has is part of the effort to rid the internet of IE6 browsers. You can read more about it http://www.theie6countdown.com/join-us.html. There is a simple script placed in the site which detects IE 6 & 7 browsers and asks users to update their software.
TYNT is a SEO tool http://www.tynt.com/. It allows webmasters to track any information which is copied and pasted from their site. It creates links back to the site, and tracks all activity on those links.
Both of those scripts are working normally and neither should negatively impact crawls. My Google WMT show my site being crawled normally, no errors. I have multiple pages ranked #1. With that said, there are plenty of opportunities for me to improve, which is why I am here.
The position of the Title and meta tags within the head should not be a factor at all. I did hear Matt Cutts share once that webmasters could move the information to the top to help if there are other issues such as a page where the tag is not properly closed, but that is not really relevant in this case.
-
Your source code looks different to me, if not .... abnormal. What's with doing above the description anyway. I don't know if that code is blocking the crawler from that code to . I'm no code monkey, but all my metas are right next to each other. Yours are completely separated from the title. Take out that code and see if it happens again. OR better yet, go to GWT and FETCH AS GOOGLEBOT. For sure that will tell you what Google bot is seeing.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Short description about our search results drop + forum moving to subdomain question.
Hello, here is our story. Our niche is mental health (psychology, psychotherapy e.t.c). Our portal has thousand of genuine articles, news section about mental health, researches, job findings for specialists, a specialized bookstore only with psychology books, the best forum in country, we thousands of active members and selfhelp topics etc. In our country (non english), our portal has been established in 2003. Since then, for more than 15 years, we were no 1 in our country, meaning that we had the best brand name, hundreds of external authors writing unique content for our portal and hundreds of no1 keywords in google search results. Actually, we had according to webmaster tools, more than 1.000 keywords, in 1 and 2 position. (we were ranking no1 in all the best keywords). Before 2 years, we purchased the best domain in our niche. I ll use the below example (of course, domains are not the real ones):
Intermediate & Advanced SEO | | dodoni
We had: e-pizza.com and now we have: pizza.com
We did the appropriate redirects but from day one, we had around 20-30% drop in search engines. After 6 months -which is something that google officialy mentions, we lost all "credits from the old domain.. .and at that point, we had another 20-30% drop in search results. Further more, in any google core update, we were keep dropping. Especially in last May (coronovirus update), we had another huge drop. We do follow seo guides, we have a dedicated server, good load speed, well structured data, amp, a great presence in social media, with more than 130.000 followers, etc. According to our investigation, we came to one only conclusion: that our forum, kills our seo (of course, noone in our team can guarantee that this is the actual reason of the uge drop in may-in coronovirus google core update). We believe that the forum kills our seo, because it produces low quality posts by members. For example, psychopharmacology in a very active sections and we believe, google is very "sensitive" in these kind of posts and information. So here is the question: although the forum is very very active, with thousands of new topics and posts every month, we are thinking of moving it to a subdomain, from the subfolder that now is.
This will help our domain authority to increase from 38 that is stuck 2 years now, to larger scales. We believe that althougth this forum gave a great boost to the portal, in the past 10-15 years, it somehow makes a negative impact now. If I could give more spesific details, I d say this: in all seo tools we run, the best kewwords bringing visitors to us, arent anymore, psychology and psychotherapy and mental health and this kind of top-keywords, but are mostly the ones from the forum, like: I want to proceed with a suicide, I m taking efexor or xanax and they have side effects, why i gain wieght with the antidepressants I get etc. 1. Moving our forum to subdomain, will be some kind of pain, since it is a large community, with thousands of backlinks that we somehow must handle in a proper way, also with a mobile application, things that will have to change and probably have some kind of negative impact. Would that be according to your knowledge a correct move and our E-A-T will benefit for google, or since google will know that the subdomain is still part of the same website/portal, it will handle it somehow, the same way as it does now? I have read hundreds of articles about forum in subdomains or in subfolders, but none of them covers a case stydy like ours, since most articles are talking about new forums and what is the best way to handle them and where is the best place to create them (in subfolder of subdomain) when from scratch. Looking forward to your answers.0 -
Question & Review should be seperate page
Hi pls look at the below page, http://www.powerwale.com/store/exide-xplore-xltz4-3ah-battery/76933 is questions and review should be in seperate page, as i think that in the future the comments, will become Key word stuffing for the product page. Pls suggest.. If yes, suggest the best url as well.. thanks
Intermediate & Advanced SEO | | Rahim1191 -
Paging Question: Rel Next or Canonical?
Hi, Lets say you have a category which displays a list of 20 products and pagination of up to 10 pages. The root page has some content but when you click through the paging the content is removed leaving only the list of products. Would it be best to apply a canonical tag on the paging back to the root or apply the prev/next tags. I understand prev/next is good for say a 3 part article where each page holds unique content but how do you handle the above situation? Thanks
Intermediate & Advanced SEO | | Bondara0 -
Questions about 301 Redirects
I have about 10 - 15 URLs that are redirecting to http://www.domainname.comwww.domainname.com/. (which is an invalid URL)The website is on a Joomla platform. Does anyone know how I can fix this? I can't figure out where the problem is coming from.
Intermediate & Advanced SEO | | JohnParker27920 -
Our keyword rank fell drastically since our last moz crawl. What gives?
Our site was just crawled last night an we had a number of keywords fall 25% or so, from say # 30 to 38. What would cause such a drastic fall? We haven't taken down pages or changed changed page titles, url or content. One thing we are having trouble with is our meta description and meta keywords for each page. Drupal 7 seems to be giving us trouble and the meta information isn't showing up anymore. Would that cause the fall? Any other suggestions?
Intermediate & Advanced SEO | | drudalton1 -
Rel Alternate tag and canonical tag implementation question
Hello, I have a question about the correct way to implement the canoncial and alternate tags for a site supporting multiple languages and markets. Here's our setup. We have 3 sites, each serving a specific region, and each available in 3 languages. www.example.com : serves the US, default language is English www.example.ca : serves Canada, default language is English www.example.com.mx : serves Mexico, default language is Spanish In addition, each sites can be viewed in English, French or Spanish, by adding a language specific sub-directory prefix ( /fr , /en, /es). The implementation of the alternate tag is fairly straightforward. For the homepage, on www.example.com, it would be: -MX” href=“http://www.example.com.mx/index.html” /> -MX” href=”http://www.example.com.mx/fr/index.html“ />
Intermediate & Advanced SEO | | Amiee
-MX” href=”http://www.example.com.mx/en/index.html“ />
-US” href=”http://www.example.com/fr/index.html” />
-US” href=”http://www.example.com/es/index.html“ />
-CA” href=”http://www.example.ca/fr/index.html” />
-CA” href=”http://www.example.ca/index.html” />
-CA” href=”http://www.example.ca/es/index.html” /> My question is about the implementation of the canonical tag. Currently, each domain has its own canonical tag, as follows: rel="canonical" href="http://www.example.com/index.html"> <link rel="canonical" href="http: www.example.ca="" index.html"=""></link rel="canonical" href="http:>
<link rel="canonical" href="http: www.example.com.mx="" index.html"=""></link rel="canonical" href="http:> I am now wondering is I should set the canonical tag for all my domains to: <link rel="canonical" href="http: www.example.com="" index.html"=""></link rel="canonical" href="http:> This is what seems to be suggested on this example from the Google help center. http://support.google.com/webmasters/bin/answer.py?hl=en&answer=189077 What do you think?0 -
Apache Mod Rewrite question
Hi everybody, I need to rewrite this url using mod rewrite, but I've got stuck. http://www.diamondgeezer.com/theultimate/search/index.php?sortprice=asc&followSearch=9673&q=eternity+rings I'd like it to show this one instead: http://www.diamondgeezer.com/eternity-rings I'm no expert on this stuff, so any help would be great! Thanks
Intermediate & Advanced SEO | | neooptic0 -
Googlebot crawling partial URLs
Hi guys, I've checked my email this morning and I've got a number of 404 errors over the weekend where Google has tried to crawl some of my existing pages but not found the full URL. Instead of hitting 'domain.com/folder/complete-pagename.php' it's hit 'domain.com/folder/comp'. This is definitely Googlebot/2.1; http://www.google.com/bot.html (66.249.72.53) but I can't find where it would have found only the partial URL. It certainly wasn't on the domain it's crawling and I can't find any links from external sites pointing to us with the incorrect URL. GoogleBot is doing the same thing across a single domain but in different sub-folders. Having checked Webmaster Tools there aren't any hard 404s and the soft ones aren't related and haven't occured since August. I'm really confused as to how this is happening.. Thanks!
Intermediate & Advanced SEO | | panini0