This course teaches students how to compute and visualize metrics from controlled experiments in a deeper way than is usually taught using a project and assessment based approach. This course will be extremely useful for anybody who needs to compute or understand experiment analytics, especially in consumer Internet. The course includes analysis exercises and a final exam. The analysis exercises will ensure a rigorous learning process for the students who have the determination to complete the course. This course will enable the student to become the most highly sought after data scientist in the company.
Check out the instructor and understand his approach to data science. Larry describes his qualifications and gives a concrete definition of 'data science.'
Walk thru of the topics you will learn in the course. This lecture ties together the curriculum into a coherent whole and explains the vision for the course.
Understand the rationale for the data used in this course. The data sets were carefully designed and constructed by the instructor to enable the students to participate in the calculations and get accurate feedback in the assessments.
Make sure you can download and use the free SQL tool. This tool is nice because you don't need to setup any database, but still benefit from the compact notation of SQL for complex data processing.
Download and test the free tool for creating visualizations and doing analysis in this course. Tableau is an excellent and relatively straightforward tool for building sophisticated interactive dashboards.
Walk through the material covered in this section. Understand how the lectures in this section build upon each other and the overall vision for creating and using statistical units in an analytics hypercube.
Use SQL to compute statistical units and key business metrics. This lecture shows you how to create the table used for self testing in the next lecture.
Use SQL to compute test metrics for input into the coding exercise. This lecture shows you how to do some basic calculations for the following assessment using the table built in the previous lecture.
Add experiment variant and other exploratory features to the statistical units. This lecture shows you how to make the table as well as run some calculations on it for input into the following assessment.
Learn how to compress the experiment analytics into a hypercube. This lecture wraps up the section and shows you how to run some calculations on the hypercube table for input into the following assessment.
This lecture walks through the material covered in this section. The lecture describes how the lectures in this section build on each other and culminate in the creation of the interactive dashboard.
Compute the trend and deviations of variant traffic in Tableau. Learn how to build a basic visualization.
Learn how to compute and visualize statistical significance of traffic fluctuations. Extract some computed numbers from Tableau for input into the following assessment.
Learn how to setup variables for dynamically computing statistical significance in Tableau, as well as other key metrics.
Learn how to visualize statistical significance in Tableau. Extract some computed numbers from Tableau for input into the following assessment.
Learn how to compute dynamically populated dimension metrics in Tableau. Extract some calculated numbers from Tableau for input into the following assessment.
Build all the waterfall metric component tables in Tableau using multiple worksheets. Understand the relationship between the waterfall metrics.
Pull together all the waterfall metric components into a dashboard. Extract some computed numbers from Tableau for input into the following assessment.
Learn how to add filters to the Tableau dashboard. Slice the waterfall metrics simultaneously by statistical unit features. Extract some calculated numbers from the dashboard for input into the following assessment.
This lecture walks through the material covered in this section. The lecture describes how the lectures in this section build upon each other and the overall vision.
Learn how to build an analytic unit feature hypercube table for each feature. This lecture gives a whiteboard description of the calculations used in the screencast in the following lecture.
This lecture shows the screencast for the queries used for creating the analytic unit feature hypercubes. The lecture also shows how to extract some numbers for input into the following assessment.
This lecture shows how to combine multiple analytic unit feature hypercube tables into a multi-cube table. A description is given of how to get some numbers for input into the following assessment.
This lecture shows a whiteboard description of how to add analytics unit feature dimensions to the experiment dashboard. This will help the student to better follow the screencast shown in the following lecture.
This lecture has a screencast showing how to add analytic unit feature dimensions to the experiment dashboard. Some calculated numbers are extracted from the dashboard for input into the following assessment.
This lecture gives a brief overview of the final exam contained in the following quiz.
The final exam is composed of 10 multiple choice questions. The first 6 questions will exercise your use of the experiment dashboard built in the course. The last 4 questions help you think about a few key concepts introduced in the course. Have fun!
This bonus lecture gives some practical advice and best practices for data science experimentation in consumer internet, assuming that the student intends to use the knowledge gained from this course in a real life application.
Larry Wai is an avid scientist who’s currently pushing the frontiers of data science. Larry has designed a unique scalable framework that allows multiple individual data scientists to become “full stack,” taking control of their own destinies from the exploration and research phase, through algorithm deployment, experiment set-up, and deep analytics. Prior to being Head of Data Science at Tubular Labs, Larry was Chief Data Scientist at Udemy, a lead data scientist at Groupon and, earlier, a principal analyst at Yahoo Search. He has patents related to search and discovery data science methods. Before moving into the consumer Internet field, he was a scientist working on particle astrophysics, with physics experiments ranging from neutrino oscillations to detection of dark matter. He was recently awarded a portion of the 2016 Breakthrough Prize in Fundamental Physics for fundamental discovery and exploration of neutrino oscillations.