Data Science with Python - A Complete Guide!: 3-in-1

This video will give you a glimpse of what you will gain throughout this course.

Before you start with a new technology or a tool, it is a good practice to get a quick introduction, overview and its benefits to gain a firm belief in its potential and capabilities. This video is just that start with Python.

• Understand what data science is

• Explore Python

• Use optional arguments

Now that you are well aware about what data science and Python are, let’s not take too long to get started. In this video, we will install the necessary packages and take a quick step to make everything ready at our hand.

• Install Anaconda

• Leverage Conda to install packages

Though we are eager to start some practical stuff, let’s spare some time to know the essential packages you need in day to day coding while working on machine learning tasks. In this video, we will give a brief introduction of several useful packages that you need on several occasions for specific tasks while solving a machine learning problem.

• Get Introduced to widely used packages

• Learn the source to download the package

• Explore the commands for installation

Jupyter deserves more than a brief presentation. In this video, we are going to delve fully in detail about its installation, and usage for data science. Let’s go ahead and do it right now!

• Explore the basics of Jupyter Notebook

• Learn about the Jupyter Magic commands

As we progress through the concepts presented in the course, in order to facilitate you understanding, learning, and memorizing processes, we will illustrate practical and effective data science Python applications on various explicative datasets. You will always be able to immediately replicate, modify, and experiment with the proposed instructions and scripts on the data that we will use in this course.

• Load the Iris dataset

• Explore the MLdata.org public repository

• Learn to load data directly from CSV or text files

In this video, we will begin with the first step that is to load and preprocess the data using pandas.

• Start with a CSV file and pandas

• Dealing with problematic data and big datasets

• Preprocess data and select data

In this video, we will learn to deal with the types of data.

• Map a categorical value to a list of numerical ones

• Deal with text classification and clustering

• Scrap the Web with Soup

In this video, we will learn the ways of creating NumPy arrays.

• Transform lists to unidimensional arrays

• Control the memory size

• Extract data from pandas

When arrays need to be manipulated by mathematical operations, you just need to apply the operation on the array with respect to a numerical constant (a scalar) or an array of the same shape.

• Learn about Matrix operations

• Perform slicing and indexing

• Perform stacking

Exploratory Data Analysis (EDA), or data exploration, is the first step in the data science process. This is what you require to understand a dataset better, check its features and its shape, validate an initial hypothesis, and get a preliminary idea about the next step that you want to pursue in the coming data science tasks.

• Load the iris dataset and use the .describe() method

• Use the .median(), .mean() and .std()methods

• Check the distribution of the feature

Sometimes, you'll find yourself in a situation when features and target variables are not really related. It's a very important step for the overall process because it completely depends on the skills of the data scientist, who is the one responsible for artificially changing the dataset and shaping the input data to better fit the learning model. Let’s see how to deal with such situations.

• Import the dataset containing house prices in California

• Apply KNN Regressor and normalize the input features using Z-scores

• Compare the regression tasks on this new feature set

Oftentimes, you will have to deal with a dataset containing a large number of features, many of which may be unnecessary. This is a typical problem and keeping only the interesting features is a way to not only make your dataset more manageable but also have predictive algorithms work better instead of being fooled in their predictions by the noise in the data. So, how could we do that? Let’s get an answer through this video.

• Compute a correlation matrix

• Use Principal Component Analysis and Latent Factor Analysis

• Use Linear Discriminant Analysis and Latent Semantical Analysis

In data science, examples are at the core of learning from data processes. If unusual, inconsistent, or erroneous data is fed into the learning process, the resulting model may be unable to correctly generalize the accommodating of any new data. How can you get rid of such situations? Let’s have a look at the solution.

• Implement Univariate outlier detection

• Use the EllipticEnvelope

• Use the OneClassSVM algorithm

In order to evaluate the performance of the data science system that you have built and check how close you are to the objective that you have in mind, you need to use a function that scores the outcome. Typically, different scoring functions are used to deal with binary classification, multilabel classification, regression, or a clustering problem. Now, let's see the most popular functions for each of these tasks and how they are used by machine learning algorithms.

• Use Multilabel classification

• Implement Binary classification

• Predict real numbers or regressions

After loading our data, preprocessing it, creating new useful features, checking for outliers and other inconsistent data points, and finally choosing the right metric, we are ready to apply a machine learning algorithm. Here you will learn how can we correctly apply the learning process in order to achieve the best model for prediction to be generally used with similar yet new data.

• Load the dataset, fit the linear SVM classifier to the data and verify the results

• Split the data into two mutually exclusive sets

• Use different hypothesis to test the accuracy

Unfortunately, relying on the validation and testing phases of samples brings uncertainty along with a reduction of the learning examples dedicated to training. A solution would be to use cross-validation, and Scikit-learn offers a complete module for cross-validation and performance evaluation. Let’s take a step further and see how to use these in our code.

• Use the three possible hypotheses for the digits dataset

• Use cross-validation iterators

• Perform sampling and bootstrapping

A machine learning hypothesis is not simply determined by the learning algorithm but also by its hyperparameters and the selection of variables to be used to achieve the best learned parameters. In this video, we will explore how to extend the cross-validation approach to find the best hyperparameters that are able to generalize to our test set.

• Import the module, set the scorer variable using a string parameter, and create a list made of two dictionaries

• Use the grid search technique

• Build a custom scoring functions

With respect to the machine learning algorithm that you are going to use, irrelevant and redundant features may play a role in the lack of interpretability of the resulting model, long training times and, most importantly, overfitting and poor generalization. Here in this video, you will explore different solutions to this problem.

• Select feature based on feature variance

• Implement Univariate selection

• Use the recursive elimination

As a concluding topic, in this video, we will discuss how to wrap together the operations of transformation and selection we have seen so far, into a single command, a pipeline that will take your data from source to your machine learning algorithm.

• Combine features together and chaining transformations

• Build custom transformation functions

In this video, we will download all required datasets and tools that will be required throughout this section.

• Load the libsvmtools datasets

• Load covertype dataset

In this video, we will predict the target value.

• Load dataset and divide it into two sections

• Train and fit the regressor in the training set

Naive Bayes is a very common classifier used for probabilistic binary and multiclass classification. We will learn more about them in this video.

• Try an example of the application of the Gaussian Naive Bayes classifier

• Measure the accuracy of the regression task by using the MAE score

In this video, we will learn about the K nearest Classifiers.

• Shuffle the observations of a dataset

• Check performance

• Check training speed

In all the methods we've seen so far, every sample or observation has its own target label or value. In some other cases, the dataset is unlabeled and, in order to extract the structure of the data, you need an unsupervised approach. In this video, we're going to introduce two methods to perform clustering, as they are among the most used methods for unsupervised learning.

• Create the artificial datasets and represent them by a plot

• Apply Kernel PCA

• Apply DBSCAN

This video provides an overview of the entire course.

This video discusses how to access data from local files in different formats. The aim of the video is to understand the most common file formats used to exchange data, and how Python makes it easy to access these formats.

• Read local files using Python

• Access common data formats like CSV and JSON

• Serialize binary data using the pickle module

This video introduces the notion of exploratory analysis and outlines some of the common steps that an analyst needs to take when dealing with a new data set.

• Compute summary statistics on a new data set

• Understand the distribution of different values

• Bucketing and plotting the data

This video discusses the most common steps that are required to get the data in the right shape, including preprocessing and cleaning.

• Identify data that need cleaning and preprocessing

• Deal with duplicates and missing data

• Transform the data

This video discusses the process of breaking a string down into individual tokens or phrases, including text data from different domains (For example, social media versus general English).

• Identify tokens from text

• Deal with text from different domains

• Identify phrases to capture more complex concepts

This video discusses the process of removing stop-words (unimportant words) and punctuation from a list of tokens.

• Define stop-words and unimportant words

• Remove stop-words and punctuation tokens

• Deal with Unicode symbols

This video introduces the most common steps for text normalization that is the process of transforming a token into its canonical form.

• Transform tokens using case normalization

• Transform tokens using stemming and normalization

• Transform tokens using synonym mapping

This video discusses how to calculate word frequencies within documents and across a whole collection, and how to read.

• Find the most common word or phrases in a document

• Find the most common word or phrases in a collection

• Understand the role of word frequencies in text analytics.

This video introduces scikit-learn as the main library for machine learning.

• Install and understand the use cases for scikit-learn

• Understand the main components in scikit-learn

• Understand how to approach a machine learning problem with scikit-learn

This video introduces regression analysis as the problem of predicting a quantity, or a continuous variable, using scikit-learn.

• Shape a problem as regression analysis

• Perform regression analysis using scikit-learn

• Evaluate the quality of our prediction using MSE

This video introduces binary classification as the problem of assigning a label to an item, out of two possible labels.

• Shape a problem as binary classification

• Perform binary classification on a data set using scikit-learn

• Evaluate the quality of a classifier

This video extends the concepts from the previous video introducing multi-class classification as the problem of assigning a label to an item, out of many possible labels.

• Shape a problem as a multi-class classification

• Perform multi-class classification using scikit-learn

• Evaluate a classifier and applying cross-validation

This video introduces clustering as the problem of grouping together similar items to find hidden structure in our data.

• Define a notion of similarity between items

• Apply a centroid-based method to the clustering problem

• Implement clustering in scikit-learn with means

This video discusses how to analyze time series data using Pandas, observing seasonality and understanding the general trend of a series.

• Understand seasonality and trend of a series

• Decompose a time series with an additive model

• Implement time series analysis with Pandas and statsmodels

This video discusses recommender systems and how to implement a movie recommendation engine using collaborative filtering.

• Understand recommendation systems as information filtering engines

• Understand collaborative filtering to exploit the “wisdom of the crowd”

• Implement and evaluating a collaborative filtering algorithm