Introduction to Natural Language Processing (NLP)

We will be using the Anaconda distribution of Python throughout this course. You can download it for free from https://www.continuum.io/downloads.

Using the Anaconda Prompt (you can search for this program after Anaconda has installed), type conda install jupyter to install Jupyter. Jupyter is a notebook style interface for interactive coding.

To launch Jupyter, open your Anaconda Prompt and type jupyter notebook. This will launch a new notebook instance in your internet browser. The Anaconda Prompt is now actively running Jupyter notebook locally. If you close the prompt, Jupyter will no longer be able to run. You can simply open the Anaconda Prompt and type jupyter notebook at any time to restart Jupyter, and pick up where you left off.

In your Windows Command Prompt (search for the "command prompt" program) type the following:

python

import nltk

nltk.download()

This will open a new window which will allow you to download extra data and packages we will be using throughout the course.

We will be using the Anaconda distribution of Python throughout this course. You can download it for free from https://www.continuum.io/downloads.

Using the Terminal (you can search for this app), type conda install jupyter to install Jupyter. Jupyter is a notebook style interface for interactive coding.

To launch Jupyter, type jupiter notebook in your Terminal window. This will launch a new notebook instance in your internet browser. The Terminal is now actively running Jupyter notebook locally. If you close that Terminal window, Jupyter will no longer be able to run. You can simply open the Terminal and type jupyter notebook at any time to restart Jupyter, and pick up where you left off.

In a new Terminal window or tab, (one that is not running Jupyter) type the following:

python

import nltk

nltk.download()

This will open a new window which will allow you to download extra data and packages we will be using throughout the course.

A python list is a variable that stores comma separated values.

mylist = ["a", "b", "c"]

A python dictionary stores key-value pairs.

d = {
'Python': 'programming',
'English': "natural",
'French': 'natrual',
'Javascript' : 'programming'
}

If we were to look up any of the keys (Python, English, French, or Javascript), we would get back the associated value.

We will often use for-loops to scan through lists.

We will use if statements to look for special conditions.

We will use functions to reuse code that we write.

We will be using the Natural Language Tool Kit (NLTK) throughout this course.

Regular expressions are a great way to find specific character patterns in your text data.

Tokenization is the act of breaking text into smaller entities. These can either be sentences (sentence tokens) or individual words (word tokens).

Sentiment analysis is the act of determining if a given collection of text is more positive or negative in nature.

Sentiment analysis is the act of determining if a given collection of text is more positive or negative in nature.

We can use Naive Bayes classification techniques to determine if a name is traditionally more masculine or feminine.

We can use Naive Bayes classification techniques to determine if a name is traditionally more masculine or feminine.

With Term Frequency-Inverse Document Frequency (TF-IDF), we can easily determine key words in documents.