Introduction to NLP

Explore rule-based, probabilistic, and deep learning approaches to NLP, and examine how morphology, syntax, semantics, and pragmatics enable tasks like text classification, language modeling, and chat.

Master tokenization methods, including sentence, word, whitespace, and treebank tokenization, and apply stop-word removal and normalization techniques like stemming and lemmatization to prepare text for NLP models.

Explore text preprocessing for natural language processing by normalizing sample text from Project Gutenberg with Python, using Google Colab to remove punctuation, filter stopwords, lowercase, stem, and lemmatize.

Examine rule-based natural language processing methods, including sentiment analysis, part-of-speech tagging, and df idf-based document classification, with emphasis on stopwords and logarithmic scale handling.

Explore a rule-based sentiment analysis workflow that classifies book reviews using positive and negative word lists, per-sentence scoring, and basic file operations.

Explore probabilistic models using word co-occurrence, bigrams and trigrams to predict the next word under the Markov assumption, and apply supervised learning to spam classification with feature vectors.

Practice session demonstrates spam classification by loading a dataset with features and a spam label, training a classifier, and predicting on a test set with about 93.8% accuracy.

Explore how word representations evolve from one-hot to distributed models using co-occurrence and PMI, addressing semantics and dimensionality. See neural-network based word prediction and vector operations reveal semantic relationships.