Tripling Your Conversion Rates With A/B TestingLars Lofgren

@larslofgren

2X and 3X in 9-12 months

KISSmetrics homepage conversion to signups.

Nov 2013 = 2.85% Aug 2014 = 8.71%

I Will Teach double confirmation page

July 2015 = 63% June 2016 = 82%

I Will Teach popup monthly lead counts

Aug 2015 = 1800 Aug 2016 = 7100

Win rates at KISSmetrics after 65 A/B testsN

umbe

r of T

ests

0

10

20

30

40

Confirmed Winner Marginal Confirmed Loser

52%20%28%

Win rates at I Will Teach after 96 A/B testsN

umbe

r of T

ests

0

15

30

45

60

Confirmed Winner Marginal Confirmed Loser

60%

23%17%

If most of our tests fail…we can’t depend on any

single test.

You’re in a quantity game. More tests = more wins.

We don’t use A/B tests to pick winners. We use them to avoid losers as we stack

confirmed winners.

Cycle through as many tests as possible while only

launching confirmed winners.

8 Rules of A/B Testing

1. Above all else, the control stands.

2. Get 2000+ people through the test within 30 days.

3. Wait at least 1 week before checking data.

4. Only launch variants at 99% statistical significance.

Fast forward a week…

5. If the test drops below 10%, ditch it.

6. If no new winner after a month, ditch variant.

7. Get the next test ready while you wait.

8. If you have another goal, say so before starting

Cycle through as many tests as possible while only

launching confirmed winners.

Q&ALars Lofgren

@larslofgrenlars@larslofgren.com

bit.ly/triple-conversions