.: Split Testing Sucks And Other Heresies
Category:Home / Business / Marketing
Split testing is what marketing is all about; right? Wrong! There is a much better way to get the answer to 95% of the kinds of questions you might consider split testing. Let’s step through it.
First of all, you must decide your desired outcome. Is it profitability? Is it search engine ranking? Is it more traffic? Is it more inbound links?
Once you have decided your desired outcome, you must find a way that OTHER PEOPLE’S sites can be measured for that outcome.
That’s right; I’m not advocating split testing because there is a much faster and even more accurate way to get the same results by simply looking at the results of OTHER PEOPLE!
It’s called statistical analysis. It’s what most scientific advancements have been based on! When you go to the doctor, does he split test antibiotics and exercise on you? Of course not! You would fire him immediately. Instead, he looks at the results of studies where other people had the same desired outcome you want (getting rid of that fever, cough, whatever) and prescribes the drug or treatment that was proven safe and effective for a statistically significant number of OTHER PEOPLE!
Will it always work for you because it worked on a statistically significant number of OTHER PEOPLE? No; of course not. But it will work a majority of the time. The fact is that split testing has the same problems. When you find an answer from split testing and choose the “a” version over the “b” version based on 20 actions… There is a percentage chance that you actually chose the wrong version. Increase the number of actions and you increase your chances of picking the right one. The same is true when looking at OTHER PEOPLE’S results instead of your own.
OK; so how can we measure some of these things. Search Engine Ranking is easy. You can compare sites at the #1 ranking with sites at the #100 ranking…. or sites in the top 10 with sites in the 101-110 range in ranking. No problem.
More traffic? You can find open logs for a statistically significant number of sites. A less perfect measurement might be Alexa or Jupiter ratings. When dealing with less accurate measurements like that, simply increase your sample size or compare more distant extremes (Alexa rankings in the top 100 vs Alexa ranking in the 99,900-100,000 traffic rankings).
More inbound links? Once again you can use open logs and look at referral entries… or you can trust the link: command at MSN… or something less reliable like the link: command at Google (once again, just increase your sample size and/or compare more distant extremes in number of inbound links).
How about profitability? This is my favorite. This is why most of us are here. I used this measurement for my Glyphius software and all of the copywriting statistical studies posted on my blog and in the Statistical Copywriting Online Home Study Course.
How can you get the profitability figures for other web-sites? Are they just going to turn them over? Actually, public companies do just that. You could use something like that public data. Also, many affiliate networks give indirect profitability figures (the marketplace at Clickbank, the EPC rating on CJ are both less accurate profitability numbers).
Or you could do what marketers have quietly done for decades before ecommerce and the Internet even existed. This is how Glyphius and the Statistical Copywriting course were created. It is really quite simple. Ask yourself these two questions:
1. If I was paying for advertising to a site that was profitable… would I continue to pay for that advertising month after month as long as it was profitable?
2. If I was paying for advertising to a site that wasn’t profitable… would I continue to pay for that advertising month after month?
The answer to #1 is clearly a “yes” for a vast majority of people. The answer to #2 is clearly a “no” for a vast majority of people. There are exceptions. Some large companies don’t even track the profitability of their ads… so they will add some incorrect data if we use this measurement.
The same thing happens in science all the time. Patients in medical studies lie or exagerate about their results (in both directions). People aren’t completely predictable. They do things against their own self-interest sometimes.
The solution? Same as with the other imperfect measurement techniques… increase the sample size and/or compare more distant extremes. If people were 100% predictable, then we wouldn’t need a very large sample size to look at these web-sites.
So, let’s take an example to see how this works. Let’s say some idiot “guru” marketer is telling you that using the digit “7″ in your pricing will increase your conversion ratio and your profitability. OK; fine. You have three choices:
1. Believe him/her. Congratulations; you have just joined the religious society of idiot “guru” marketers. Instead of embracing science for those things that can be easily researched, you have decided to apply a religious belief to such a topic instead. You can feel a little better about yourself because 98% of all people are in the same boat. Unfortunately 98% of all people will retire at 65 years old broke.
2. Split test it. Arrgh! You know why you haven’t already done this… it takes forever to get statistically significant results. Then you know that there are still hundreds or thousands of things you still need to split test after that.
3. Statistically analyze OTHER PEOPLE’S pricing and their profitability.
Look; some things need to be split tested. Still; wouldn’t it be nice if you could come very close to the real result before you even start your split tests? Do you really need to split test a bright purple background on your site when it’s obvious that a vast majority of profitable sites use a white background? Of course not! Test only those things that can’t be easily found by doing a statistical analysis of OTHER PEOPLE’S sites!
OK; so how do we get the above answer?
First we make us a list of profitable and unprofitable sites. We already know how to do that; right? No? OK; let’s break it down.
First we search for a statistically significant number of keywords on Google or any other search engine that shows paid advertising (or we go get some magazines that advertise web-sites… or we go get a nickle ad sheet and find some classifieds that include URLs… it doesn’t matter… just go to a source of paid advertising that shows URLs).
Then we make a list showing the exact ad and the URL. Now we wait. How long do we wait? We wait long enough to have some folks with unprofitable ads change them or remove them. If you used magazines, wait 6 months. Some magazines have a policy of only allowing you to run an ad for three months at a time. So, it takes more than 3 months before you see unprofitable ads disappear. On Google, you can probably get away with waiting just a couple of weeks. The longer you wait, the fewer sites you will have to compare (the old comparing more distant extremes rule).
Now you do it again. Go make a list of ads and their URLs that are still paying for advertising.
Now compare your two lists. Wherever you find an ad that is exactly the same months later, you have an ad that is very likely to be profitable. Put that URL on your “profitable” list. Put the other URLs that were on your first list, but are missing on your second list on your “unprofitable” list.
Cool; now you have a list of profitable and unprofitable sites. You can now run all kinds of quick statistical analysis on these sites. Since you can review a hundred sites much, much faster than you can usually get 100 results in a split test… you are way ahead now.
Now, simply go through your “proftiable” list and figure out the percentage of prices that have a “7″ in them. Now go through your unprofitable list and figure out the percentage of prices that have a “7″ in them. Compare the two numbers. Are they very close to each other? Then using a “7″ probably doesn’t matter very much in your pricing when it comes to profitability. Are they very distant? Did you find that 83% of unprofitable sites used a “7″ in their price, but only 21% of profitable sites did? Woo hoo! You have found your answer in just an hour instead of waiting for days for a split test.
So how do you know if you had a statistically significant number of sites to test? Yikes; with something this complex, that would take a couple of years of statistical training to figure that out. Let me give you a real world way of finding out that will save you from the hell of majoring in statistics in college (although if you are a single male, I highly recommend doing the major… for some reason statistics chicks are much better looking than chicks taking other majors… statistically speaking).
Predictability. Turn around your study and do a split test. What percentage of the time did the results of the split test match the predicted outcome of the statistical analysis. The closer you get to 100%, the more you can rely on the dataset of profitable and unprofitable sites you built. Of course, 0% isn’t the bottom of the scale here… 50% is. So if you are only a bit over 50%, then you realy need to look at the assumptions of your study… and the sample size… but if you are well over 75%, then you have a decent dataset that you can rely on for future studies.
You might be thinking… wait! I still had to do a split test! And worse… building that list of profitable and unprofitable sites took me just as long as it would take to do a split test… so what gives?
You only have to build the list of profitable and unprofitable sites and validate it once! Then you can do an unlimited number of studies using the same data!
The same is true for any measurement. My ranking data asks several thousand questions using just one set of data of high ranking and low ranking sites (with a 91% confidence rating)! I’ve answered dozens of copywriting questions using my one dataset of profitable and unprofitable sites (with a 96% confidence rating in that case).
Yes; it’s a lot of work just to get the answer to the “7″ question…. but then it is very little work to then ask “what about the digit ‘3′”? Or “Are red headlines more profitable or blue headlines?” or “Is long copy really more profitable or not?”
Instead of wasting your time doing a split test for days for each answer (or if you are a newbie still struggling to get traffic… maybe months to get an answer) and during that time not making as much money as you could be making… you can just spend an hour tabulating the results from your profitable and unprofitable lists of sites! Then you can have an answer that you can count on 96% of the time.
Guess what? With 20 actions in a split test (20 sales in this case for both “a” and “b”), you only have a 95% confidence in your results. How long will it take you to get at least 20 sales for both your “a” and your “b” version in a split test?
So there you go. I’m a heretic. I hate split testing my own sites for answers I can easily get by analyzing OTHER PEOPLE’S sites. It isn’t the first time I disagree with 98% of marketing “gurus” and it probably won’t be the last.
Article keywords: marketing, sales, ecommerce, copywriting, webmaster, business, statistics
Article Source: http://www.articles32.com
James D. Brausch is the creator of several software and information products about copywriting, traffic generation and product creation. His Statistical Copywriting Online Home Study Course ($65) can be found here: www.StatisticalCopywriting.com
.: New Marketing Articles
1). Birkenstocks & The Customizable Art Of Footwear
Birkenstocks are a German brand of quality footwear, which is best known for it’s contoured footbed. When most people purchase a new pair of shoes, they expect to have a ‘break-in’ time. This usually lasts for a few days and, at that time, the shoe is pretty comfortable and broken in for everyday wear. The main difference with Birkenstocks is that they approach the ‘break-in’ time much differently.
2). Millionaire Marketing in 3 Small Steps
What is money? A child might say that it’s something rich people have and poor people don’t. An engineer might go into a detailed and lengthy explanation, but we all know that money is what really “makes the world go around.”
In the business world, money is the definition of success or failure. That’s why it’s vitally important that every entrepreneur has a good understanding of money – even if their business is still struggling to get off the ground.
3). Free T Shirt Offer – Are they really free?
The modern day craze has created a huge demand for designer clothing all over the world. Many young guns are going for a casual wear than what it used to be 20 years ago. A simple T-shirt and low waist jeans makes both the genders comfortable. It’s become almost a compulsory act to hang out in t-shirts and jeans even in colleges. Youths like to have wide collection of t-shirt of current fashion.
4). How To Promote Your Home Based Business On A Shoestring Budget!
Getting customers is the key to success in any business. There are many ways to promote your home based business - the only way to know which ones are best for your business is to test. Here are some tips for getting customers without spending a fortune!
1. Tell everyone you know that you are starting a business AND ask them to tell everyone they know.
5). Cash in on Neighbors
What could be more welcoming than the spicy smell of hot apple pie to receive you into a brand new neighborhood full of people you’ve never met? Hopefully you life has been full of warm, hospitable neighbors wherever you’ve lived. Perhaps not…
If you’ve ever had a bad experience with a neighbor, you know just how much those invisible people really know about you.
6). Technology Controlled Marketing
RSS feeds and blogs took Internet marketing by a storm in 2005. Every time we turn around there’s a brand new marketing vehicle speeding down the Internet highway. In the fast paced race of the virtual world, it’s easy to get swept away in the current of technology.
If we’re honest with ourselves, we’ll admit that computers have taken control of both our personal world and the business world.
7). Finding Your Niche In The Business World
Who am I and what do I love to do? Well, isn't this the twenty million dollar question! A more appropriate question might be "Who was I and what did I love to do?"
As you search to find yourself, and what it is you love to do, you may find the task harder than you thought it would be. Try to think about your childhood. Can you remember what thrilled you as a child? Like many people you've probably forgotten what brought try joy and excitement to your essence.
.: Top Marketing Articles
1). John Warwicker - Co-Founder of Tomato Design Company
Design company Tomato
( http://www.tomato.co.uk ) was founded in 1991 in London by John Warwicker, Steve Baker, Dirk Van Dooren, Karl Hyde, Richard Smith, Simon Taylor and Graham Wood. In 1994, Michael Horsham and Jason Kedgely joined.
tomato specializes in: Architectural Design, Consultancy, Drawing, Education, Electronic Interactive Media, Film & Commercial Direction, Graphic Design, Fashion, Motion Graphics, Music & Sound, Strategy, Branding & Identity, Photography, Publishing, Title Sequences, Typography, Writing.
2). The Benefits Of Branding
Branding is the process of creating distinctive and durable perceptions in the minds of consumers. A brand is a persistent, unique business identity intertwined with associations of personality, quality, origin, liking and more. Here’s why the effort to brand your company or yourself pays off.
1. Memorability. A brand serves as a convenient container for a reputation and good will.
3). Ultimate Wealth Package: Is It Worthy Of Its Name?
What Is Ultimate Wealth Package All About?
Have you been considering the Ultimate Wealth Package? The sales page makes a lot of promises, but doesn't give you much detail at all. So let me fill you in!
Ultimate Wealth Package is a guide that has been put together by a young guy named Mark Warren. Within the guide he shows you in simple steps how he himself made his internet fortune, and how you too, by following the same steps can do the same.
4). MySpace Marketing
With the ever-growing popularity of internet social groups, MySpace has become the leader. MySpace has become so popular; it has spawned a whole market of new business opportunity through marketing and advertising. No other website in the world allows one person the ability to communicate with millions of people with the click of a button. This has allowed e-commerce a new channel of communication with potential customers.
5). Eniva: The Next Billion Dollar Home Based Business?
Delray Beach, FL – While tirelessly searching for the perfect home based business opportunity, a Florida man stumbles across a real, passionate, solid, and trusting eight year old company called the ENIVA Corporation.
After researching the company, the products, and the founders of ENIVA, Brisson soon realized the potential of this fast growing Minnesota based company: the potential to be the next billion dollar home based business opportunity.
6). Subliminal Persuasion
Subliminal persuasion? It is simply influencing people at a level below their conscious recognition. Many people don't even realize they are being influenced by a smile, making even that a subliminal technique. Here are two more subtle methods.
Subliminal Persuasion Using Inflection
It is easy to assume that a sentence like "I can't promise you that price.
7). Avoiding A "Failure to Communicate"
It's in every classified ad and most job descriptions: must have excellent communication skills. Worse yet, once hired, it reappears annually at review time: Improve communication skills. What's a worker to do? Communication skills don't require a graduate degree, just some common sense, a bit of homework and a better understanding of the role communication plays — in everything:
Listen.