Please help to identify the following bots and spiders
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you help by advising how to stop a URL from referring to another URL on my website please?
Stopping a redirect from one URL to another due to a 404 error? Referred URL which is (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/%5Bnull%20id=43484%5D) Referring URL (https://webwritinglab.com/know-exactly-what-your-ideal-clients-want-in-8-easy-steps/)
Technical SEO | | Nichole.wynter20200 -
How google bot see's two the same rel canonicals?
Hi, I have a website where all the original URL's have a rel canonical back to themselves. This is kinda like a fail safe mode. It is because if a parameter occurs, then the URL with the parameter will have a canonical back to the original URL. For example this url: https://www.example.com/something/page/1/ has this canonical: https://www.example.com/something/page/1/ which is the same since it's an original URL This url https://www.example.com/something/page/1/?parameter has this canonical https://www.example.com/something/page/1/ like i said before, parameters have a rel canonical back to their original url's. SO: https://www.example.com/something/page/1/?parameter and this https://www.example.com/something/page/1/ both have the same canonical which is this https://www.example.com/something/page/1/ Im telling you all that because when roger bot tried to crawl my website, it gave back duplicates. This happened because it was reading the canonical (https://www.example.com/something/page/1/) of the original url (https://www.example.com/something/page/1/) and the canonical (https://www.example.com/something/page/1/) of the url with the parameter (https://www.example.com/something/page/1/?parameter) and saw that both were point to the same canonical (https://www.example.com/something/page/1/)... So, i would like to know if google bot treats canonicals the same way. Because if it does then im full of duplicates 😄 thanks.
Technical SEO | | dos06590 -
Help: domain name change and Google News
Hi. I work for a regional news source, and our (separate) Spanish-language news publication recently changed its domain name. The publication lost its Google News inclusion. Most of their traffic came from Google News, so traffic tanked. They're trying to get back in. They reapplied but didn't get approved. They're now in the 30-day waiting period to reapply again. The website is run by a third-party company, which handled the domain name change in April (2015). That company has been running their site for a couple of years. Our in-house devs' hands are tied on helping, because we (at the mother company) don't manage their site. This third party has not been responsive. The Spanish pub folks have reached out to me to help them prepare for Round 2 of reapplication. I'm the mothership in-house SEO, but I've never experienced this situation before. Because everything seems to be in order besides the ham-handed changes, my best advice to them so far is: You'll have to wait until Google gets to know you again, unfortunately. Does that sound right? Any pointers out there for bringing their best possible A-game to the next round?
Technical SEO | | christyrobinson1 -
Yoast WordPress SEO settings please help
Hello 🙂 Can you please look at these screenshots of my Yoast WordPress SEO settings http://www.zaslike.com/files/h5149mi5435dspiswfm.jpg http://www.zaslike.com/files/5dlhmjxfh2j0hqswesha.jpg http://www.zaslike.com/files/fmx1pwih240gwiofh86s.jpg http://www.zaslike.com/files/w7tyvlhgr5vhv149b9a.png http://www.zaslike.com/files/l9lo37jfpeqmrpufke8.png are they good ? Do i need to change something or correct ? please help Thank you !!!! :))))
Technical SEO | | wolfinjo0 -
Need Help writing 301 redirects in .htaccess file
SEOmoz tool shows me 2 errors for duplicate content pages (www.abc.com and www.abc.com/index.html). I believe, the solution to this is writing 301 redirects I need two 301 redirects 1. abc.com to www.abc.com 2. /index.html to / (which is www.abc.com/index.html to www.abc.com) The code that I currently have is ................................................... RewriteEngine On
Technical SEO | | WebsiteEditor
RewriteCond %{HTTP_HOST} ^abc.com
RewriteRule (.*) http://www.abc.com/$1 [R=301,L] Redirect 301 http://www.abc.com/index.html http://www.abc.com ...................................................... but this does not redirect /index.html to abc.com. What is wrong here? Please help.0 -
Internal links - is div on click still no followed by google?
Hi Mozzers Does anyone know if are still no followed by Google From a UX perspective, making a container div clickable will work well, but i don't want this link to absorb any link juice as text within the div would make much better anchor text, so i would rather that link was receiving the juice. Is the above the best approach to this issue of UX vs SEO? Many thanks Justin
Technical SEO | | JustinTaylor880 -
How to do a no follow on site search
We have a site search that is causing a huge amount of errors as the SEOmoz crawler is showing these as duplicate content. Our first thought was to do a no-follow on the site-search directory, but we realized that the site search is /site-search.aspx and URl strings appear at the end for hundreds of pages. How dow we/how can we no-follow an undetermined amount of URL strings?
Technical SEO | | Apptixweb0 -
Keywords and content and seo advice please
Hi i am building a site at the moment which i am working on. please ignore the state of the site as we are just playing with designs at the moment but please do take notice of the top part. The site is a travel magazine but i am a bit concerned. The keywords that we will be looking at to drive travel will be as follows, cheap flights gatwick, holiday magazine and travel magazine. Now i do not want the site to look untidy with loads of content describing the site but at the same time i want google to know what the site is and for people to pick the site up with the search terms that we are aiming for. the site is www.cheapflightsgatwick.com Can anyone please show me some examples of how i should structure the site to attract the keywords and give me some advice. It seems hard when you are designing a magazine site where the content changes all the time to try and attract the search engines with your keywords. any advice would be great
Technical SEO | | ClaireH-1848860