Learning Path: From Python Programming to Data Science

This video provides an overview of the entire course.

The aim of this video is to introduce us to Python.

We will learn how to collect and store the data.

We will explore how to collect and store twitter tweets.

We will talk about database design.

We will explore Pandas and other databases.

We will explore the concepts of Panda series, data frames and columnar operations.

We will take a look operations and how to exactly work with columns.

We will explore how to merge various operations and learn how to export data to JSON/CSV.

We will take a look at what exactly arrays are, their different types, and histogram functions.

See exactly what simple aggregations are.

We will explore the concept of linear algebra.

We will learn how to present stories via simple visualizations and representations.

We will learn the different types of graphical representations.

We will learn how to create Simple XY plots and axis scales.

We will learn how to handle text data.

We will learn how to classify words.

We will take a look at stemming of words.

We will use the simple sentiment analysis using scrapped tweets.

We will learn how to group dimensions and also take a look at the different types of data that is generated.

We will take a look at New metrics and dimensions will be derived to get hidden insights.

We will take a look at the concept of co-relation analysis.

We will briefly go over what we covered in the course and also take a glimpse at what the future holds for us.

This section gives an overview of the entire course.

Importing data from csv into Python can be a bit tricky. It needs careful inspection and appropriate functions. Let's see how we can do that.

When we are automating a data pipe for many files, we are not in a position to convert an Excel file into CSV and then import it. This video shows us how to import data directly from an Excel file.

We've learned how to import data from CSV and Excel. But how do we do that with a file that has fixed-width data? Let's explore.

Although tab-delimited format is simple to read as csv files, we need to ensure that certain parameters are there to keep the reading process accurate. Let's explore how we can do that.

Let's explore how we can import data from a JSON resource like GitHub, and How to get it and process it later.

Modern applications often hold different datasets inside relational databases or other databases like MongoDB, and we have to use these databases to produce beautiful graphs. This video will show us how to use SQL drivers from Python to access data.

Data coming from the real world needs cleaning before processing or even visualization. It's not fully automated and we need to understand outliers in order to clean the data. Let's see how we can do that.

In scientific computing, images are often represented as NumPy array data structures. We can import images using various techniques. In this video, we will take a look at using image processing in Python, mainly related to scientific processing and less on the artistic side of image manipulation.

In this video, we will see different ways of generating random number sequences and word sequences. Some of the examples use standard Python modules, and others use NumPy/SciPy functions.

Data that comes from different real-life sensors is not smooth; it contains some noise that we don't want to show on diagrams and plots. In this video, we introduce a few advanced algorithms to help with cleaning of data coming from real-world sources.

There are different plots used for representing data differently. In this video, we'll compare them and understand advanced concepts in data visualization. We would also plot sine and cosine plots and customize them.

Now that we've learned the concepts of basic plotting and customizing, this video will show us a variety of useful axis properties that we can configure in matplotlib to define axis lengths and limits.

There are different kinds of audiences to whom the data is presented. Having lines set up distinct enough for target audiences for example, vivid colors for young audience leaves a great impact on the viewer. This video shows how we can change various line properties such as styles, colors, or width.

As we now know how to change various line properties such as styles, colors, and width, this video will guide us with adding more data to our figure and charts by setting axis and line properties.

Legends and annotations explain data plots clearly and in context. By assigning each plot a short description about what data it represents, we enable an easier model for the viewer. This video will show how to annotate specific points on our figures and how to create and position data legends.

Spines define data area boundaries; they connect the axis tick marks. There are four spines. We can place them wherever we want. As they are placed on the border of the axis, we see a box around our data plot. This video will demonstrate how to move spines to the center.

Histograms are often used in image manipulation software as a way to visualize image properties such as distribution of light in a particular color channel. This video will help us create histograms in 2D.

To visualize the uncertainty of measurement in our dataset or to indicate the error, we can use error bars. Error bars can easily give an idea of how error free the dataset is. In this video, we will see how to create bar charts and how to draw error bars.

Pie charts are special in many ways, the most important being that the dataset they display must sum up to 100 percent or they are just not valid. Let's explore how we can create pie charts to represent data in a better way.

The matplotlib library allows us to fill areas in between and under the curves with color so that we can display the value of that area to the viewer. In this video, we will learn how to fill the area under a curve or in between two different curves.

If you have two variables and want to spot the correlation between those, a scatter plot may be the solution to spot patterns. This type of plot is also very useful as a start for more advanced visualizations of multidimensional data. Let's see how to create a scatter plot.

To be able to distinguish one particular plot line in the figure, we need to add a shadow effect.

Adding a data table beside our chart helps to visualize information.

You can create custom subplot configurations on your plots in this video.

To spot differences in patterns and compare plots visually in the figure, we need to customize our grids.

To display isolines, we create contour plots.

To distinguish clearly between two different plots, we fill the areas with different patterns.

When the information is radial in nature, we need a polar plot to display information.

You will learn how to visualize a real-world task in this video.

You must be curious to plot 3D data after getting your hands on 2D. Python provides a toolkit called mplot3d in matplotlib for this. Let's go ahead and explore its working!

Similar to 3D bars, you might want to create 3D histograms since these are useful for easily spotting correlations between three independent variables. Let us now dive into it!

This video will walk you through graphics rendering with OpenGL. So let's go ahead and do it!

Images can be used to highlight the strengths of your visualization in addition to pure data values. It maps deeper into the viewer's mental model, thereby helping the viewer to remember the visualizations better and for a longer time. Let's see how we could use them in Python!

This video will walk you through how you can make simple yet effective usage of the Python matplotlib library to process image channels and display the per-channel histogram of an external image.

The best geospatial visualizations are done by overlaying data on the map. This video will show you how to project data on a map using matplotlib's Basemap toolkit. Let's dive into it!

This video will take you through the generation of random images to tell humans and computers apart. Let's do it!

With the logarithmic scale, the ratio of consecutive values is constant. This is important when we are trying to read log plots. Let us step ahead and see how to perform it!

In this video we will discuss how to create a stem plot which will display data as lines extending from a baseline along the x-axis.

In this video we will visualize wind patterns or liquid flow, and we will use uniform representation of the vector field for this. So, let's go ahead and do it!

Color-coding the data can have great impact on how your visualizations are perceived by the viewer, as they come with assumptions about colors and what colors represent. This video will walk you through the steps showing the use of colormaps!

If we want to take a quick look at the data and see if there is any correlation, we would draw a quick scatter plot.Iin this video, you will understand scatter plots.

If you have two different datasets from two different observations, you want to know if those two event sets are correlated. You want to cross-correlate them and see if they match in any way. This video will let you achieve this goal!

How you could predict the growth of stock dividends? In this video we will dive into some interesting steps which will let you understand the importance of autocorrelation for this prediction!

Let's look into how to visualize two-dimensional vector quantities such as speed and direction of wind!

How will you visually compare several similar data series? This video will walk you through making a box-and-whisker plot which achieves this goal!

One form of very widely used visualization of time-based data is a Gantt chart. Let us see how to work with it!

Error bars are useful to display the dispersion of data on a plot. So, let's explore their use in Python for data visualization.

This video will let you explore more features of text manipulation in matplotlib, giving a powerful toolkit for even advanced typesetting needs. Let's dive into it.

This video will explain some of the programming interfaces in matplotlib and make a comparison of pyplot and object-oriented API. Let us now explore it!

Machine learning algorithms need processed data for operation. Let’s explore how to process raw data in this video.

Algorithms need data in numerical form to use them directly. But we often label data with words. So, let’s see how we transform word labels into numerical form.

Linear regression uses a linear combination of input variables to estimate the underlying function that governs the mapping from input to output. Our aim would be to identify that relationship between input data and output data.

There are some cases where there is difference between actual values and values predicted by regressor. We need to keep a check on its accuracy. This video will enable us to do that.

Linear regressors tend to be inaccurate sometimes, as outliers disrupt the model. We need to regularize this. We will see that in this video.

Linear model fails to capture the natural curve of datapoints, which makes it quite inaccurate. So, let’s go through polynomial regressor to see how we can improve that.

Applying regression concepts to solve real-world problems can be quite tricky. We will explore how to do it successfully.

We don’t really have an idea on which feature contributes to the output and which doesn’t. It becomes critical to know that, in case we’ve to omit one. This video will help you compute their relative importance.

There might be some problems where the basic regression methods we’ve learned won’t help. One such problem is bicycle demand distribution. You will see how to solve that here.

Evaluating the accuracy of a classifier is an important step in the world of machine learning. We need to learn how to use the available data to get an idea as to how this model will perform in the real world. This is what we are going to learn in this section.

Despite the word regression being present in the name, logistic regression is actually used for classification purposes. Given a set of datapoints, our goal is to build a model that can draw linear boundaries between our classes. It extracts these boundaries by solving a set of equations derived from the training data.

Bayes’ Theorem, which has been widely used in probability to determine the outcome of an event, enables us to classify the data in a smarter way. Let us use its concept to make our classifier more amazing.

While working with data, splitting data correctly and logically is an important task. Let’s see how we can achieve this in Python.

In order to make splitting of dataset more robust, we repeat the process of splitting with different subsets. If we just fine-tune it for a particular subset, we may end up over fitting the model, which may fail to perform well on unknown data. Cross validation ensures accuracy in such a situation.

When we want to fine-tune our algorithms, we need to understand how the data gets misclassified before we make these changes. Some classes are worse than others, and the confusion matrix will help us understand this.

Let's see how we can apply classification techniques to a real-world problem. We will use a dataset that contains some details about cars, such as number of doors, boot space, maintenance costs, and so on, to analyze this problem.

Let’s see how the performance gets affected as we change the hyperparameters. This is where validation curves come into the picture. These curves help us understand how each hyperparameter influences the training score.

Learning curves help us understand how the size of our training dataset influences the machine learning model. This is very useful when you have to deal with computational constraints. Let's go ahead and plot the learning curves by varying the size of our training dataset.

Let’s see how we can build a classifier to estimate the income bracket of a person based on 14 attributes.

Building regressors and classifiers can be a bit tedious. Supervised learning models like SVM help us to a great extent. Let’s see how we can work with SVM.

There are various kernels used to build nonlinear classifiers. Let’s explore some of them and see how we can build a nonlinear classifier.

A classifier often gets biased when there are more datapoints in a certain class. This can turn out to be a big problem. We need a mechanism to deal with this. Let’s explore how we can do that.

Let’s explore how we can train SVM to compute the output confidence level of a new datapoint when it is classified into a known category.

It’s critical to evaluate the performance of a classifier. We need certain hyper parameters to do so. Let’s explore how to find those parameters.

Now that we’ve learned the concepts of SVM thoroughly, let’s see if we can apply them to real-world problems.

We’ve already used SVM as a classifier to predict events. Let’s explore whether or not we can use it as a regressor for estimating traffic.

The k-means algorithm is one of the most popular clustering algorithms, which is used to divide the input data into k subgroups using various attributes of the data. Let’s see how we can implement it in Python for Clustering data.

Vector quantization is popularly used in image compression, where we store each pixel using fewer bits than the original image to achieve compression.

The Mean Shift is a powerful unsupervised learning algorithm that's used to cluster datapoints. It considers the distribution of datapoints as a probabilitydensity function and tries to find the modes in the feature space. Let’s see how to use it in Python.

Many a times, we need to segregate data and group them for the purpose of analysis and much more. We can achieve this in Python using theagglomerative clustering. Let’s see how we can do it.

In supervised learning, we just compare the predicted values with the original labels to compute their accuracy. In unsupervised learning, we don't have any labels. Therefore, we need a way to measure the performance of our algorithms. Let’s see how we could evaluate their performance.

Wouldn't it be nice if there were a method that can just tell us the number of clusters in our data? This is where Density-Based Spatial Clustering of Applications with Noise (DBSCAN) comes into the picture. Let us see how we can work with it.

How will we operate with the assumption that we don't know how many clusters there are. As we don't know the number of clusters, we can use an algorithm called Affinity Propagation to cluster. Let's see how we can use unsupervised learning for stock market analysis with this.

What could we do when wedon't have labeled data available all the time but it's important to segment the market so that people can target individual groups? Let’s learn to build a customer segmentation model for this situation.

One of the major parts of any machine learning system is the data processing pipeline. Instead of calling functions in a nested way, it's better to use the functional programming paradigm to build the combination. Let's take a look at how to combine functions to form a reusable function composition.

The scikit-learn library has provisions to build machine learning pipelines. We just need to specify the functions, and it will build a composed object that makes the data go through the whole pipeline. Let’s see how to build it in Python.

While working with the training dataset, we need to make a decision based on the number of nearest neighbors in it. This can be achieved with the help of the NearestNeighbor method in Python. Let’s see how to do it.

When we want to find the class to which an unknown point belongs, we find the k-nearest neighbors and take a majority vote. Let's take a look at how to construct this.

A good thing about the k-nearest neighbors algorithm is that it can also be used as a regessor. Let’s see how to do this!

In order to find users in the database who are similar to a given user we need to define a similarity metric. Euclidean distance score is one such metric that we can use to compute the distance between data points. Let’s look at this in more detail in this video.

The Euclidean distance score is a good metric, but it has some shortcomings. Hence, Pearson correlation score is frequently used in recommendation engines. Let's see how to compute it.

One of the most important tasks in building a recommendation engine is finding users that are similar. Let's see how to do this in this video.

Now that we’ve built all the different parts of a recommendation engine, we are ready to generate movie recommendations. Let’s see how to do that in this video.

With tokenization we can define our own conditions to divide the input text into meaningful tokens. This gives us the solution for dividing a chunk of text into words or into sentences. Let's take a look at how to do this.

During text analysis, it's useful to extract the base form of the words to extract some statistics to analyze the overall text. This can be achieved with stemming, which uses a heuristic process to cut off the ends of words. Let's see how to do this in Python.

Sometimes the base words that we obtained using stemmers don't really make sense. Lemmatization solves this problem by doing things using a vocabulary and morphological analysis of words and removes inflectional word endings. Let's take a look at how to do this in this video.

When you deal with a really large text document, you need to divide it into chunks for further analysis. In this video, we will divide the input text into a number of pieces, where each piece has a fixed number of words.

When we deal with text documents that contain millions of words, we need to convert them into some kind of numeric representation so as to make them usable for machine learning algorithms. A bag-of- words model is what helps us achieve this task quite easily.

The goal of text classification is to categorize text documents into different classes. This is an extremely important analysis technique in NLP. Let us see how we can build a text classifier for this purpose.

Identifying the gender of a name is an interesting task in NLP. Also gender recognition is a part of many artificial intelligence technologies. Let us see how to identify gender in Python.

How could we discover the feelings or sentiments of different people about a particular topic? This video helps us to analyze that.

With topic modeling, we can uncover some hidden thematic structure in a collection of documents. This will help us in organizing our documents in a better way so that we can use them for analysis. Let’s see how we can do it!

Reading an audio file and visualizing the signal is a good starting point that gives us a good understanding of the basic structure of audio signals. So let us see in this video how we could do it!

Audio signals consist of a complex mixture of sine waves of different frequencies, amplitudes and phases. There is a lot of information that is hidden in the frequency content of an audio signal. So it’s necessary to transform the audio signal into a frequency domain. Let’s see how to do this.

We can use NumPy to generate audio signals. As we know, audio signals are complex mixtures of sinusoids. Let’s see how we can generate audio signals with custom parameters.

Music has been explored since centuries and technology has set new horizons to play with it. We can also create music notes in Python. Let’s see how we can do this.

When we deal with signals and we want to use them as input data and perform analysis, we need to convert them into frequency domain. So, let’s get hands-on with it!

A hidden Markov Model represents probability distributions over sequences of observations. It allows you to find the hidden states so that you can model the signal. Let us explore how we can use it to perform speech recognition.

This video will walk you through building a speech recognizer by using the audio files in a database. We will use seven different words, where each word has 15 audio files. Let’s go ahead and do it!

Let’s understand how to convert a sequence of observations into time series data and visualize it. We will use a library called pandas to analyze time series data. At the end of this video, you will be able to transform data into the time series format.

Extracting information from various intervals in time series data and using dates to handle subsets of our data are important tasks in data mining. Let’s see how we can slice time series data using Python.

You can filter the data in many different ways. The pandas library allows you to operate on time series data in any way that you want. Let's see how to operate on time series data.

One of the main reasons that we want to analyze time series data is to extract interesting statistics from it. This provides a lot of information regarding the nature of the data. Let’s see how to extract these stats.

Hidden Markov Models are really powerful when it comes to sequential data analysis. They are used extensively in finance, speech analysis, weather forecasting, sequencing of words, and so on. We are often interested in uncovering hidden patterns that appear over time. Let’s see how we can use it.

The Conditional Random Fields (CRFs) are probabilistic models used to analyze structured data and also to label and segment sequential data. Let us see how we can use it to work on our input dataset!

This video will get you hands-on with analyzing stock market data and understanding the fluctuations in the stocks of different companies. So let’s see how to do this!

OpenCV is the world's most popular library for computer vision. It enables us to analyze images and do a lot of stuff with it. Let’s see how to operate it!

When working with images, it is essential to detect the edges to process the image and perform different operations with it. Let’s see how to detect edges of the input image in Python.

The human eye likes contrast! This is the reason that almost all camera systems use histogram equalization to make images look nice. This video will walk you through the use of histogram equalization in Pyhton.

One of the essential steps in image analysis is to identify and extract the salient features for the purpose of computer vision. This can be achieved with a corner detection technique and SIFT feature point in Python. This video will enable you to achieve this goal!

When we build object recognition systems, we may want to use a different feature detector before we extract features using SIFT; that will give us the flexibility to cascade different blocks to get the best possible performance. Let’s see how to do it with Star feature detector.

Have you ever wondered how you could build image signatures? If yes, this video will take you through creating features by using visual codebook, which will enable you to achieve this goal. So, let’s dive in and watch it!

We can construct a bunch of decision trees that are based on our image signatures, and then train the forest to make the right decision. Extremely Random Forests (ERFs) are used extensively for this purpose. Let’s dive in and see how to do it!

While dealing with images, we tend to tackle problems with the contents of unknown images. This video will enable you to build an object recognizer which allows you to recognize the content of unknown images. So, let’s see it!

Webcams are widely used for real-time communications and for biometric data analysis. This video will walk you through capturing and processing video from your webcam.

Haar cascade extracts a large number of simple features from the image at multiple scales. The simple features are basically edge, line, and rectangle features that are very easy to compute. It is then trained by creating a cascade of simple classifiers. Let’s see how we can detect a face with it!

The Haar cascades method can be extended to detect all types of objects. Let's see how to use it to detect the eyes and nose in the input video.

Principal Components Analysis (PCA) is a dimensionality reduction technique that's used very frequently in computer vision and machine learning. It’s used to reduce the dimensionality of the data before we can train a system. This video will take you through the use of PCA.

What if you need to reduce the number of dimensions in unorganized data? PCA, which we used in the last video, is inefficient in such situations. Let us see how we can tackle this situation.

When we work with data or signals, they are generally received in a raw form. Or rather we can say they are a mixture of some unwanted stuff. It is essential for us to segregate them, so as to work on these signals. This video will enable you to achieve this goal.

We are now finally ready to build a face recognizer! Let’s see how to do it!

Let’s start our neural network adventure with a perceptron, which is a single neuron that performs all the computations.

Now that we know how to create a perceptron, let's create a single-layer neural network which will consist of multiple neurons in a single layer.

Let’s build a deep neural network, which will have multiple layers. There will be some hidden layers between the input and output layers. So, let us explore it.

Let us see how we can use vector quantization in machine learning and computer vision.

This video will walk you through analyzing sequential and time series data and enable you to extend generic models for them.

Let us look at how to use neural networks to perform optical character recognition to identify handwritten characters in images.

Let's build a neural-network-based optical character recognition system.

3D visualization is very important in data representation. So we need to learn the simple yet effective method of plotting 3D plots.

You are going to learn to plot bubble plots in this video.

Sometimes we just need to write a code that runs close to the metal. Cython gives us that option.

When there are distribution tables and various labels, pie charts are handy to express data.

To keep a track of data with respect to time, we need to plot date-formatted time series data.

When we need to compare data of two different entities, we need to plot histograms. This video is going to help you do that.

Heat maps are useful when data in two groups is associated point by point.

When we visualize real-time signals, it becomes imperative to animate the dynamic signals so that they are updated continuously. This video will help you do that.

This video will provide an overview of the course.

What is Deep Learning, and when is it the way to go?

How to avoid programming Deep Learning from scratch? Let’s take a look at it in this video.

How to get our first deep neural network trained?

How are neural networks trained?

How can we avoid making a differentiation of functions and make backpropagation easier?

How do Keras and other libraries use Theano work behind the scenes?

How does Keras work?

How does Keras work? How does one write a basic, fully connected neural network layer in Keras?

Understand what convolutional neural networks are and how to use them. How can we write convolutional layers with Python?

How can we solve complex image datasets (for example, cats versus dogs) without training a full model from scratch?

How does Keras work?

We will solve complete image datasets with pretrained models: classifying cats versus dogs.

How to write a loop in Theano?

How can one define neural network layers with internal states?

Recurrent or convolutional: How can one know which layer they should use?

How can we classify sentiments from text?

How can we automatically describe an image in English?

What is TensorFlow?