Optimization & A/B Testing Statistics
3.0 (40 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
1,137 students enrolled
Wishlisted Wishlist

Please confirm that you want to add Optimization & A/B Testing Statistics to your Wishlist.

Add to Wishlist

Optimization & A/B Testing Statistics

Speed up your a/b tests by doing it right from the start.
3.0 (40 ratings)
Instead of using a simple lifetime average, Udemy calculates a course's star rating by considering a number of different factors such as the number of ratings, the age of ratings, and the likelihood of fraudulent ratings.
1,137 students enrolled
Created by Jared Waxman
Last updated 11/2013
English
Current price: $10 Original price: $35 Discount: 71% off
1 day left at this price!
30-Day Money-Back Guarantee
Includes:
  • 3 hours on-demand video
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Have a coupon?
Description

Whether you've got a lean startup or a fat Fortune 500, the faster you learn the faster you'll grow.  Optimization and a/b testing is at the heart of learning fast.

I guarantee you will learn something in this course that will raise your skill level.  With the 30-day money-back guarantee, you can't lose. 

We start with the basics, then cover the 8 steps of running a solid a/b test.  Next we dive deep into the statistics behind hypothesis testing.  In the long-run you will save your organization headaches by setting up tests correctly and analyzing them with the right statistical rigour. 

There is double and triple digit ROI around optimization for companies that figure it out.  Start now and impress your colleagues on Monday morning.

Topics include:

Examples of a/b tests

Hypothesis testing

Measurement as risk reduction

Selecting a KPI or success metric

8 Steps for Running an A/B Test

Selecting from amongs a/b test and MVT test designs

Lift Threshold

Null Hypothesis

Statistical significance

Sample size estimates

confidence interval

test statistic

t-tests

standard error of the mean

chi-square

Fischer Exact test

Statistical Power

Type I error

Type II error

p-values

How to choose what statistical test to run

Students Who Viewed This Course Also Viewed
Curriculum For This Course
Expand All 21 Lectures Collapse All 21 Lectures 02:48:36
+
Introduction
5 Lectures 24:35

Madmen of today should be called mA/Bmen since they ply experimentation, not hard liquor from 9 to 5.

Learn why and how to apply a/b testing to your marketing efforts.  We'll start at the beginning, with the foundations of the science behind hypothesis testing.

Preview 01:54

A quick intro what organizations gain from optimization programs and where it can be applied.

Preview 03:32

We cover two different a/b tests to provide a starting point for what a test really looks like.

Preview 03:53

We cover some of the special language of optimization.  A good reference to come back to.

Preview 04:59

You probably know there is a difference.  But we apply this to optimization and explore the options.

Correlation & Causation
10:17
+
Goals & Test Designs
4 Lectures 45:08

Get this wrong and the hard work that went into the experiment is lost.   Choosing appropriate success metrics in an A/B test is a bit more nuanced than many think.

Select the KPI
16:55

How do you setup your test when you care about optimizing to more than one outcome?

Multiple KPIs
12:33

We cover the basic test designs you might consider when planning out your experiment and the pros and cons of each.

Test Designs
06:07

What is Measurement vs Hypothesis Testing?

Measurement
09:33
+
Testing in 8 Steps
4 Lectures 30:37
8 Steps to A/B Testing
08:11

Writing a precise statement is Step 1.

Step 1 In Depth
06:16

Determing Sample Size needed.

Step 2 In Depth
11:17

Run until sample size reached.

Step 3 In Depth
04:53
+
Statistical Tests
7 Lectures 59:42
One Sample T-Test
04:25

Two Sample T-Test
07:37

Standard Error & Confidence Intervals
14:28

Learn to select the right statistical test for your experiment. (Part 1)

Select Stat Test (part 1)
06:57

Select Stat Test (part 2)
10:47

Chi Square (part 1)
09:59

Chi Square (part 2)
05:29
+
Wrap-Up
1 Lecture 08:34
Conclusion
08:34
About the Instructor
Jared Waxman
2.7 Average rating
55 Reviews
1,448 Students
2 Courses
The Marketer's Mythbuster: A/B Testing & beyond

Jared Waxman works at Yahoo! in Sunnyvale CA where he serves as Senior Director of Growth.

Previously he worked for Adobe in San Jose CA, where he served as Group Manager for Online Optimization & Analytics for Adobe.

In his nearly twenty years of experience in Silicon Valley where he gained experience in website optimization, web analytics, and growth hacking he has been on both the tools-side and the client-side, beginning as Director of Product Management for Alexa Internet's web measurement division, before working with Amazon and Intuit. He holds a Bachelors of Science from Yale University.