How long do I split test

Adrianhenry · 19th Feb 2015, 01:07 PM

How long should I keep a split test running before the results become somewhat reliable?

100 clicks to each page? more? less?

savidge4 · 19th Feb 2015, 03:47 PM

As I see it... the math behind A/B testing is Bells Theorem. Mathematically speaking the theorem falls into its "best" stride right at 1000 views per side of your test. All of the numbers just fall into that sweet spot where a definitive yes or no A is better than B answer can be given.

That's my answer and Im sticking to it! LOL

BudiT · 19th Feb 2015, 09:43 PM

Wow. Never heard of using Bell's theorem before. That's cool! Usually I just go 100 clicks for each side.

savidge4 · 19th Feb 2015, 10:53 PM

Originally Posted by BudiT

Wow. Never heard of using Bell's theorem before. That's cool! Usually I just go 100 clicks for each side.

Oh jeeze really? If you look at A/B Significance Test you can insert your numbers. and it is using bells theorem to figure out the overall significance of you test. If the numbers are to close, on the far right column last 2 lines it will tell you basically yes trust the results, or no, not enough data.

Notice when you go to the page it is set for? 1000 LOL

Here is another link that has many different calculators: http://www.evanmiller.org/ab-testing/
including the top left one that helps you determine the sample size needed to run a decently accurate test.

Adrianhenry · 19th Feb 2015, 11:37 PM

Originally Posted by savidge4

Oh jeeze really? If you look at A/B Significance Test you can insert your numbers. and it is using bells theorem to figure out the overall significance of you test. If the numbers are to close, on the far right column last 2 lines it will tell you basically yes trust the results, or no, not enough data.

Notice when you go to the page it is set for? 1000 LOL

Here is another link that has many different calculators: Evan's Awesome A/B Tools - sample size calculator, A/B test results, and more
including the top left one that helps you determine the sample size needed to run a decently accurate test.

Thanks! Those are some great resources

richpage75 · 20th Feb 2015, 04:18 AM

Ideally, you will need at least 1000 unique visitors per week to the page that you want to test - otherwise it will take too long to get a test result (many weeks).

And its not just about traffic too - you need to get enough 'conversions' (sales, leads, signups, subscribers etc) to get a valid test result. Ideally you will need at least 25 conversions per test variation (50 conversions per week in total).

You should also take a full week to do the test - as this will help take into account variances in traffic and differences in traffic by day (weekend versus weekday etc), and this also allows enough time for fluctuations in test results to settle down.

And if you don't have enough traffic to do testing, don't give up - there are still some good ways to test with lower traffic - I created a guide on A/B testing with low traffic to help with this.

Hope that helps! Good luck with your testing...

CPABeyondNetwork · 21st Feb 2015, 11:25 AM

Split test until you have enough data and then just scale the best campaign.

Best regards,

Affiliate Manager at CPABeyond

Adrianhenry · 22nd Feb 2015, 04:16 PM

Originally Posted by CPABeyondNetwork

Split test until you have enough data and then just scale the best campaign.

Best regards,

Affiliate Manager at CPABeyond

I think this may be my favourite reply ever.

How long do I split test for?

"until you have enough data"

ahhhh thanks, how did I miss that!

razorz18 · 23rd Feb 2015, 01:29 AM

Test when you see clear winner. Its depend on your existing traffic. 100 visitors/clicks maybe not enough if you have 10k-20k per days.

Just be practical on it.

savidge4 · 23rd Feb 2015, 10:24 AM

Originally Posted by razorz18

Test when you see clear winner. Its depend on your existing traffic. 100 visitors/clicks maybe not enough if you have 10k-20k per days.

Just be practical on it.

Im going to be THAT guy. there is nothing practical about A / B testing. it is what it is, a scientific method of comparison. I can not even begin to think how many times there was a "clear winner" early in testing and then once the full testing ran its course that "winner" ended up being the loser.

So I keep pretty good notation on all of the tests I run. I just went and looked over the past 300 test that I have run. 68% of these tests had an early leader. out of those 204 tests with early leaders, 172 of those ended up being losers.

that means to bail on the process early would have had me place the least converting option 84% of the time. Jeeze... that's bad! LOL

quinng123 · 23rd Feb 2015, 01:37 PM

After I have at least 300 unique views daily, I do split testing for a month.

This has generally worked out for me for my own experiences

savidge4 · 23rd Feb 2015, 03:18 PM

Originally Posted by quinng123

After I have at least 300 unique views daily, I do split testing for a month.

This has generally worked out for me for my own experiences

Right now you are split testing at 4500 impressions per side. Not a bad thing don't get me wrong. But if you mathematically only really need say 1000 per test variable, you could effectively be running 4.5 Different tests per month. Or one heck of a A B C D E F G H test LOL

lestatega · 23rd Feb 2015, 10:07 PM

I think that this is an endless game You always will compete and weed out the bad ones. You stop when you are in the range of 30% conversion rate.

FreedomBlogger · 23rd Feb 2015, 10:21 PM

The more the better! .. and as some recommended, at least 1000 clicks. Then track every single thing and look at the numbers to see what's working best!

I hope this helps!

Keep up the great work!

I wish you the best!

Cheers!

19th Feb 2015, 01:07 PM	#1
Adrianhenry HyperActive Warrior War Room Member Join Date: 2012 Location: UK Posts: 318 Thanks: 56 Thanked 105 Times in 80 Posts	How long do I split test Share on: How long should I keep a split test running before the results become somewhat reliable? 100 clicks to each page? more? less?
	[SIGPIC][/SIGPIC] Learn AngularJS By Building A Quiz App From Scratch

19th Feb 2015, 03:47 PM	#2
savidge4 Midnight Oil Warrior Join Date: 2013 Location: Bridgeport, WV. Posts: 6,782 Thanks: 3,590 Thanked 6,601 Times in 3,490 Posts	Re: How long do I split test Share on: As I see it... the math behind A/B testing is Bells Theorem. Mathematically speaking the theorem falls into its "best" stride right at 1000 views per side of your test. All of the numbers just fall into that sweet spot where a definitive yes or no A is better than B answer can be given. That's my answer and Im sticking to it! LOL
	Success is an ACT not an idea

19th Feb 2015, 09:43 PM	#3
BudiT Overdriven Warrior Join Date: 2014 Location: Singapore Posts: 138 Thanks: 80 Thanked 51 Times in 39 Posts	Re: How long do I split test Share on: Wow. Never heard of using Bell's theorem before. That's cool! Usually I just go 100 clicks for each side.

20th Feb 2015, 04:18 AM	#6
richpage75 Conversion Rate Optimizer Join Date: 2013 Location: Cologne Posts: 29 Thanks: 0 Thanked 9 Times in 8 Posts	Re: How long do I split test Share on: Ideally, you will need at least 1000 unique visitors per week to the page that you want to test - otherwise it will take too long to get a test result (many weeks). And its not just about traffic too - you need to get enough 'conversions' (sales, leads, signups, subscribers etc) to get a valid test result. Ideally you will need at least 25 conversions per test variation (50 conversions per week in total). You should also take a full week to do the test - as this will help take into account variances in traffic and differences in traffic by day (weekend versus weekday etc), and this also allows enough time for fluctuations in test results to settle down. And if you don't have enough traffic to do testing, don't give up - there are still some good ways to test with lower traffic - I created a guide on A/B testing with low traffic to help with this. Hope that helps! Good luck with your testing...
	Rich Page http://www.rich-page.com \| Twitter: @richpage Author of 'Website Optimization: An Hour a Day'

21st Feb 2015, 11:25 AM	#7
CPABeyondNetwork HyperActive Warrior Join Date: 2015 Posts: 402 Thanks: 1 Thanked 64 Times in 60 Posts	Re: How long do I split test Share on: Split test until you have enough data and then just scale the best campaign. Best regards, Affiliate Manager at CPABeyond

23rd Feb 2015, 01:29 AM	#9
razorz18 HyperActive Warrior Join Date: 2011 Posts: 249 Thanks: 2 Thanked 40 Times in 34 Posts	Re: How long do I split test Share on: Test when you see clear winner. Its depend on your existing traffic. 100 visitors/clicks maybe not enough if you have 10k-20k per days. Just be practical on it.
	Ways to Increase Website Traffic in 2019

23rd Feb 2015, 01:37 PM	#11
quinng123 Work Hard Play Hard Join Date: 2011 Location: Los Angeles, CA Posts: 183 Thanks: 14 Thanked 39 Times in 24 Posts	Re: How long do I split test Share on: After I have at least 300 unique views daily, I do split testing for a month. This has generally worked out for me for my own experiences

23rd Feb 2015, 10:07 PM	#13
lestatega Warrior Member Join Date: 2014 Posts: 24 Thanks: 14 Thanked 5 Times in 5 Posts	Re: How long do I split test Share on: I think that this is an endless game You always will compete and weed out the bad ones. You stop when you are in the range of 30% conversion rate.

23rd Feb 2015, 10:21 PM	#14
FreedomBlogger internetmarketingblog101 Join Date: 2014 Location: Virginia - USA Posts: 820 Thanks: 201 Thanked 292 Times in 232 Posts	Re: How long do I split test Share on: The more the better! .. and as some recommended, at least 1000 clicks. Then track every single thing and look at the numbers to see what's working best! I hope this helps! Keep up the great work! I wish you the best! Cheers!
	At the beginning, I thought making money online with a blog was super super hard. Not anymore. Learn the art of making money online blogging - step by step - HERE.