Response based campaigns are sales or marketing campaigns where specific offers or advertisements are shown and we need to measure response rates per group. In this business problem we are choosing ad-copy and price points and are trying to pick the best combination. This is a traditional statistical problem and is often organized into what are called "A/B tests." However, more and more business professionals are required to directly plan and manage these sales and marketing campaigns without statistical or technical assistance.
This course will teach you how to plan and manage a response based marketing campaign. Learn:
The method we are teaching is based on Bayesian statistical inference, so is a bit different than the more traditionally taught methods (t-tests, p-values). The advantage is the a number of natural business questions (such as how much money is at risk) are easier to encode in the Bayesian framework.
The course is unfortunately a very technical topic. To mitigate this we have wrapped the complete method in an interactive worksheet that we are sharing freely. The idea is: you supply the wisdom and judgement, the worksheet can perform all of the messy calculations.
The course structure is as follows:
The methods taught in this course should greatly improve your ability to plan and manage sales and marketing campaigns. You should be able to judge if this is the course for you from the free lecture.
The course largely describes a free Shiny app that performs campaign calculations for you. The app has a planning sheet and an evaluation sheet.
The purpose of the planning sheet is to use the user inputs (prior bounds on conversion rate and conversion value) to estimate an acceptable absolute error in campaign value. This acceptable error rate is used to campaign sizes that ensure the campaign chosen has a good probability of being close to the best choice in terms of relative error. A good way to use this is to enter two different rates you wish to be able to distinguish between on the planning sheet.
The evaluation sheet shows the posterior distribution of the (unknown and unobserved) true values of the campaigns conditioned on the observations the user has input.
This lecture lays out the entire course. We define what problem we are going to solve (estimating the true response rate of your sales campaigns) and how we are going to solve it (with a free interactive worksheet). The free app is linked in the lesson extras/external-resources area. Also be sure to watch the introduction video (the video that plays if you click on the image when you first load this course).
In this lecture we will review some common technical terms from statistics. The goal is to make these terms less of a surprise when we use them.
This lecture is the meat of the course. We show how to work some problems in the campaign planning and testing worksheets. We freely share the worksheet and freely share some guide pages as downloadable materials.
Try your hand at planning a sales campaign
Now that we have worked a few exercises together we review the statistical theory behind the worksheet.
This optional slide deck gives pointers to the source code, references, and more.
I am principal at with the data science consulting firm Win-Vector LLC. Win-Vector LLC specializes in data science research, implementation, and training. I have over 10 years of experience in research, teaching, machine learning, and data science.
I am co-author of the popular book Practical Data Science with R, and I blog often on mathematics, programming, machine learning, and optimization on the Win-Vector blog.
My profesional experience includes managing a data science group for Shopping dot com (an eBay company), working in price optimization for Rapt (acquired by Microsoft), and apply machine learning at a web-scale for Kosmix (acquired by Walmart online). My original fields of study were mathematics (AB UC Berkeley) and computer science (Ph.D. Carnegie Mellon) with a heavy emphasis on probability theory.