Breaking into Data Science & Machine Learning with Python

This is a short overview of the Section.

Learning Python is like learning a new language (human spoken language like German, Spanish). In this lecture we use the analogy of human language to understand and learn Python.

This lecture will help prepare writing Python codes. This lecture starts with Python's vocabulary (keywords) and grammar then explain how to write code in python. From a very simple code this lecture will explain the key components of Python language. 

You will learn how to take input form the user and how to print output programmatically. Also you will learn basic arithmetic operation  with some primitive data types in Python.

In this session you will learn: 

1. How to manipulate String.

2. String operations.

3. Different usages of string type data.

In this lecture, we covered how to work with numeric data types in Python.

In this lecture, we covered how to work with List in Python. How to create a list, and access different list element has been discussed.

Tuple is very unique data types in Python. In this lecture we discussed how to define tuple and access tuple elements. We also discussed the "immutability" concept.

In this lecture we discussed how to work with dictionary in Python.

This is an extension of previous lecture.

Boolean data type has been discussed in this lecture.

We have discussed few examples of boolean data type in this lecture.

In this lecture, we discussed how to write conditional statements in Python. if else statement has been discussed with examples.

In this lecture we discussed how to write For and while loop in Python.

Writing function in Python has been discussed in this lecture.

Exploratory data analysis (EDA) has been introduced in this lecture. This is just a section level introductory lecture.

This is lecture is a brief explanation on how to read data from flat files. To be specific, in this video only CSV (comma separated values) is covered. For this video please keep data and notebook in the same folder.

This lecture is detail explanation of reading flat files in two scenarios,

1. When data and code in the same directory

2. When data and code in different directory

This lecture also covers,

1. How to check data for sanity

2. Different data types in pandas data frame columns

3. Basic of missing values

This is to read/load data from a excel file to pandas data frame.

This lecture is on how to read/load data from a SQL database.

First, we will create a database and then we will pull data from that database.

This lecture involves creating a database which is 80% of the length of this session, and rest of the session is pulling data.

I would recommend to take your time and understand each of this steps. As a beginner if you think the database creation part is complicated you can skip and only focus on pulling data from the database.

This lecture is on how to read/load data from a web API (Application Peripheral Interface).

This lecture covers following topics,

1. Introduction to data sanity check

2. Why domain knowledge is important in data analysis

3. How to conduct quick sanity check on the data

4. What are the technical steps of detail sanity check

5. How to look for missing values

6. How to summarize the statistics of  missing values

This lecture covers,

1. How to load data

2. How to conduct basic sanity check

3. How to investigate missing values

4. How to clean data and how to impute missing values

This is an exercise on data cleaning to make sure that we understood the last two modules on the data sanity check and data cleaning. We will learn about data cleaning by going through this exercise in detail.

First, I want you to try out this exercise by yourself. Then I will walk you through the solution in coming several lectures.

This lecture is to work together towards solving data cleaning exercise.

Pandas apply function is explained in this lecture.

This second part of the solution of the exercise on data cleaning.

This is detail lecture on looking data from various angle by doing sub setting (row and column wise). I would recommend to take break  as needed while watching this lecture.

This lecture covers

1. Data Subset by Row

2. Data Subset by Column

3. Creating Boolean mask for row selection

4. Explanation of view vs hard copy of data frame

5. Processed Data Exporting to Disk

This lecture continues on exploratory data analysis (know as EDA).

Let's try some coding by yourself. This is a good exercise on EDA.

In order to complete the exercise you need pandas group by operation. In this lecture I will explain how pandas group by operation works and will implement it in Jupiter notebook.

Now we know pandas group by function. So let's try to solve the exercise together in this lecture.

This is a introductory lecture for data visualization. We discuss about importance of data visualization in this lecture.

Simplest of all curve is the line. This lecture demonstrates how to plot lines using matplotlib and how to configure them for desired outcome.

For data visualization different types of charts/plots have been discussed in this lecture. Also discuss about when should we use these different charts. In next few lecture we will be discussing how to plot different types of charts using different libraries like matplotlib, seaborn etc.

This session is to show how we can plot categorical variables.

Box plot in different practical aspects has been discussed in this lecture.

In this lecture We demonstrated how to plot and work with violin plots.

In this lecture we discussed bar plots and violin plots.

A data scientist may encounter spatial data in their professional life. Spatial data has 'space' related component in it and this is why need map to visualize them. In this lecture we demonstrate how to visualize spatial data.

This lecture is continuation of previous lecture on spatial data visualization. We will enhance data visualization we made in the last lecture.

Stock market, weather data, financial data, sales data all has one thing in common - time component. In this lecture we will demonstrate common techniques a data scientist will use to visualize time series data.

This lecture is a continuation of previous lecture on time series data. We will use seaborn library to plot time series data.

This lecture is a continuation of previous lecture on time series data. We will use plotly library to plot time series data. Please refer to plotly installation instruction if you don't have plotly already installed or having difficulty installing it by yourself.

In this instructional video we will show how to install Plotly for high quality data visualization.

This is a introductory lecture on data wrangling/manipulation.

Often time as a data person you will be asked to slice and dice data. In this lecture we will learn how to slice data.

This lecture is continuation of slicing data.

This lecture is continuation of slicing data part 2.

This lecture is continuation of slicing data part 3.

This lecture is continuation of slicing data part 4 and we will go much deeper into the matter.

This lecture is continuation of slicing data part 5 and the last in the series.

A very important concept in data analysis is aggregation. In this lecture we will learn how to aggregate data using pandas aggregation functions.

Let's try this exercise to check our learning on aggregation.

Let's code together to solve the aggregation exercise.

Reshaping data in the desired format is important for down stream consumption of the data. We will learn how to reshape data using pandas pivot function.

Beyond pivot we can also reshape data using stacking. In this lecture we will learn how to reshape data using stacking.

This lecture is to show how we can unstack data.

When we have a situation where we have multiple data tables or data frame and we want to join them together. This is the lecture to learn how to merge multiple data into one table/data frame using pandas merge, join and concat functions.

Now that we have learned multiple different techniques on data reshaping, let's try this exercise!

In this lecture let's code together to solve the exercise.

Everyone talks about machine learning. I know you have been waiting a long time for this lecture. Let's jump into it without further delay. This is just a intro with code!

It is important to know the types of ML algorithms out there. Also to know when to use which one. In this lecture we discuss different types of ML algorithms.

This is a introductory lecture on linear regression.

To understand complex machine learning models it is better to start with linear regression. In this lecture we will discuss linear regression with code.

This lecture is continuation of previous lecture on linear regression.

To evaluate a model we need to know metrics. This lecture is on model metrics and how to calculate them using sklearn library.

Let's try to solve a problem using linear regression. This exercise is a good opportunity to build first machine learning algorithm.

Before even building model it is better to explore data. In this lecture let's do some EDA on the data set.

Once we have enough understanding of the data after doing some EDA it is time to do feature engineering. Basically coming up with some derived feature from the raw data column.

Okay, now is the time to build the model. In this lecture we will implement linear regression model based on the features we have discussed in the earlier lecture.

Since you have tried the exercise by yourself, let's try it together now!

This is a introductory lecture on logistics regression with some coding.

Sigmoid function is important to understand logistic regression. I will explain the math behind this function and also we will use coding to make things more clear.

After sigmoid function now is time to learn about logit function. In this lecture I will explain logit functions with coding.

This lecture is on logistic regression model building.

Once we have the model, now is the time to evaluate it. Let's code together to calculate those matrices.

This is the continuation of previous lecture on model performance matrices.

This lecture is on explaining accuracy of the model with example and codes.

This lecture is on explaining confusion matrix of the model with example and codes. Hopefully you will never get confused with confusion matrix!

This lecture is on explaining precision and recall of the model with example and codes.

This lecture is on explaining F-1 score of the model with example and codes.

This lecture is on explaining ROC/AUC of the model with example and codes.

This lecture summarizes all the model performance matrices in one place.

Cross validation is an important concepts in machine learning. We will explain it with code in this lecture.

When we have multiple models, how do we choose one over other. In this lecture we will learn some technique to choose model.