How to hold a variable constant for an A/B test
-
For example, let's say you want to A/B test a title tag change. You are hoping to identify whether a title tag change increases CTR. But, position is always fluctuating a bit and that affects CTR, too. So, I'm interested in how you could hold position constant in order to isolate the change in CTR that is due to the title tag change. Does anyone know of resources/tools/tutorials for how to do this?
It's been... a very long time since I took statistics (-: I have access to Excel, MS Access, and R studio.
-
Thanks for the reply. I'll check out the links you included.
I thought it might be helpful to show an example of what got me thinking along these lines. This is a study about A/B testing of title tag changes. They don't say they accounted for position fluctuation; maybe they didn't, but it seems like you'd have to in order to have meaningful results.
-
I don't see a way of doing this accurately. With organic rank fluctuations, seasonality, competition it's near impossible to "hold" a position. However, if you have a position in serps for a specific keyword with little fluctuation and get decent traffic you can establish this as your baseline. Once the title is updated and appears in the serps, begin monitoring w/search console, analytics or etc. and see if you see any decent jumps. Some links for additional information: Finding the ROI of Title tag changes using Google's CausalImpact R package, SEO Split Testing & A Beginner’s Guide to A/B Testing: Effective SEO Landing Pages. Would love to hear if anyone else has a better way.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
301 Redirect Only Home Page/Root Domain via Domain Registrar Only
Hi All, I am really concerned about doing a 301 redirect. This is my situation: Both Current and New Domain is registered with a local domain registrar (similar to GoDaddy but a local version) Current Domain: Servers are pointing to Wix servers and the website is built and hosted with Wix I would like to do a 301 redirect but would like to do it in the following way with a couple of factors to keep in mind: 99% of my link are only pointed to the home page/root domain only. Not to subdirectories. New Domain: I will register this with wix with a new plan but keep the exact sitemap and composition of current website and launch with new domain. Current Domain: I want to change server pointing to wix to point to local domain registrar servers. Then do a 301 redirect for only the home page/root domain to point to the new domain listed with wix. So 301 is done via local registrar and not via Wix. Another point to mention is it will also change from Http to Https as well as a name change. Your comments on the above will be greatly appreciated and as to whether there is risk in trying to do a 301 redirect as above. Doing it as above it also cheaper if I do the 301 via the wix platform I will need to register a full new premium plan and run it concurrently to the old plan whereas if I do it as mentioned above will only have the additional domain annual fee. Look forward to your comments. Mike
Intermediate & Advanced SEO | | MikeBlue10 -
What does Disallow: /french-wines/?* actually do - robots.txt
Hello Mozzers - Just wondering what this robots.txt instruction means: Disallow: /french-wines/?* Does it stop Googlebot crawling and indexing URLs in that "French Wines" folder - specifically the URLs that include a question mark? Would it stop the crawling of deeper folders - e.g. /french-wines/rhone-region/ that include a question mark in their URL? I think this has been done to block URLs containing query strings. Thanks, Luke
Intermediate & Advanced SEO | | McTaggart0 -
Pure Manual Spam v/s New website
My website nowwhatstudio.com hit by google pure spam and google applied manual spam action to the website. Why google applied manual spam action to my website? How Can I recover my website. I create new website (nowwhatmoments.com) with the same content from the old spam action website (nowwhatstudio.com). As google removed my old website content from search indexed. Can I use the same content for a new website? I already spend allot of money on content. If I delete my old website from the server, after that Can I use the old website content for a new website? Or Can make edits the old website content and make it 80% original for a new website? I can use my old website (nowwhatstudio) email id and social accounts for a new website?
Intermediate & Advanced SEO | | bondhoward0 -
Mobile Search Results Include Pages Meant Only for Desktops/Laptops
When I put in site:www.qjamba.com on a mobile device it comes back with some of my mobile-friendly pages for that site(same url for mobile and desktop-just different formatting), and that's great. HOWEVER, it also shows a whole bunch of the pages (not identified by Google as mobile-friendly) that are fine for desktop users but are not supposed to exist for the mobile users, because they are too slow. Until a few days ago those pages were being redirected for mobile users to the home page. I since have changed that to 404 not founds. Do we know that Google keeps a mobile index separate from the desktop index? If so, I would think that 404 should work.. How can I test whether the 404 not founds will remove a url so they DON'T appear on a mobile device when I put in site:www.qjamba.com (or a user searches) but DO appear on a desktop for the same command.
Intermediate & Advanced SEO | | friendoffood0 -
Indexing falling/search queries the same - concerned
Hello, I posted abou this a few days ago but didn't really get anywhere and now have new information after looking into it more. This is my site - http://www.whosjack.org My page indexing has been falling steadily daily currently from thousands of pages indexed to just a couple of hundred. My search queries don't seem to be currently affected, I have done crawl tests to see if the site can be crawled and put the site:whosjack.org into Google and had 12,000 results come back when goole has said it has indexed 133 and falling. However all pages indexed on the site:whosjack.org search seem to be stories with just two words in the title? I am sure I am missing out on traffic here but can't work out what the issue is and how to fix it. I have no alerts on my dashboard and when I submit sitemaps to webmaster tools I get 15,115 URLs submitted 12,088 URLs indexedwhich cant be bad?Any help/suggestions really appreciated.
Intermediate & Advanced SEO | | luwhosjack0 -
Does Google Index an Alert Div w/Delayed Hide
We have a div at the top of a client's the page that displays an alert to the user. After 30 seconds it is rendered hidden. Does Google index this? Does Google take this into account when it ranks the page?
Intermediate & Advanced SEO | | WEOMedia0 -
Keyphrase / Keyword arrangement
Hi all, What are your thoughts on the arrangement of keyphrases / words? For example, does it make a difference if the words are arranged in the following way: "Keyword 1 Keyword 2" or "Keyword 2 Keyword 1" Both ways make a phrases which is favourable in the search engines. Can I stick with 1 way or should I be going with both arrangements. Hope that is clear 🙂
Intermediate & Advanced SEO | | wtfi0 -
Microdata / Schema.org and HTTPS
I have a quick question regarding Microdata / Schema.org files that are not hosted on secure connections. I receive a receive a security error from my e-commerce site because the code references the schema over HTTP instead of HTTPS.<div< span="">itemscope itemtype="http://schema.org/Product"></div<>This is not the first time I have run into this issue. We also use MRSS schema for an RSS feed from yahoo and the same thing happens.<div< span="">xmlns:media="http://search.yahoo.com/mrss"></div<>The problem mainly lies in the fact that these schemas are not hosted over HTTPS. If you add HTTPS to the beginning of both you will get a security error.Just wondering if anyone else has dealt with this or similar issue and what the "best practices" are around this?Is it ok to obtain the schema directly and then host it on our server, over our secure connection?Thanks!
Intermediate & Advanced SEO | | AnthonyMangia0