Optimization & A/B Testing Statistics
3.3 (152 ratings)
1,492 students enrolled

# Optimization & A/B Testing Statistics

Speed up your a/b tests by doing it right from the start.
3.3 (152 ratings)
1,492 students enrolled
Created by Jared Waxman
Last updated 11/2013
English
English [Auto]
Current price: \$23.99 Original price: \$34.99 Discount: 31% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
• 3 hours on-demand video
• Full lifetime access
• Access on mobile and TV
• Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
Description

Whether you've got a lean startup or a fat Fortune 500, the faster you learn the faster you'll grow.  Optimization and a/b testing is at the heart of learning fast.

I guarantee you will learn something in this course that will raise your skill level.  With the 30-day money-back guarantee, you can't lose.

We start with the basics, then cover the 8 steps of running a solid a/b test.  Next we dive deep into the statistics behind hypothesis testing.  In the long-run you will save your organization headaches by setting up tests correctly and analyzing them with the right statistical rigour.

There is double and triple digit ROI around optimization for companies that figure it out.  Start now and impress your colleagues on Monday morning.

Topics include:

Examples of a/b tests

Hypothesis testing

Measurement as risk reduction

Selecting a KPI or success metric

8 Steps for Running an A/B Test

Selecting from amongs a/b test and MVT test designs

Lift Threshold

Null Hypothesis

Statistical significance

Sample size estimates

confidence interval

test statistic

t-tests

standard error of the mean

chi-square

Fischer Exact test

Statistical Power

Type I error

Type II error

p-values

How to choose what statistical test to run

Course content
Expand all 21 lectures 02:48:36
+ Introduction
5 lectures 24:35

Madmen of today should be called mA/Bmen since they ply experimentation, not hard liquor from 9 to 5.

Learn why and how to apply a/b testing to your marketing efforts.  We'll start at the beginning, with the foundations of the science behind hypothesis testing.

Preview 01:54

A quick intro what organizations gain from optimization programs and where it can be applied.

Preview 03:32

We cover two different a/b tests to provide a starting point for what a test really looks like.

Preview 03:53

We cover some of the special language of optimization.  A good reference to come back to.

Preview 04:59

You probably know there is a difference.  But we apply this to optimization and explore the options.

Correlation & Causation
10:17
+ Goals & Test Designs
4 lectures 45:08

Get this wrong and the hard work that went into the experiment is lost.   Choosing appropriate success metrics in an A/B test is a bit more nuanced than many think.

Select the KPI
16:55

How do you setup your test when you care about optimizing to more than one outcome?

Multiple KPIs
12:33

We cover the basic test designs you might consider when planning out your experiment and the pros and cons of each.

Test Designs
06:07

What is Measurement vs Hypothesis Testing?

Measurement
09:33
+ Testing in 8 Steps
4 lectures 30:37
8 Steps to A/B Testing
08:11

Writing a precise statement is Step 1.

Step 1 In Depth
06:16

Determing Sample Size needed.

Step 2 In Depth
11:17

Run until sample size reached.

Step 3 In Depth
04:53
+ Statistical Tests
7 lectures 59:42
One Sample T-Test
04:25
Two Sample T-Test
07:37
Standard Error & Confidence Intervals
14:28

Learn to select the right statistical test for your experiment. (Part 1)

Select Stat Test (part 1)
06:57
Select Stat Test (part 2)
10:47
Chi Square (part 1)
09:59
Chi Square (part 2)
05:29