Unsupervised Deep Learning in Python

Pca stands for principal components analysis and helps transform high-dimensional data into useful features for downstream algorithms, and also enables visualization in two or three dimensions.

Rotate data with a pca orthonormal transformation to preserve vector magnitudes and angles, effectively rotating the coordinates while keeping lengths intact.

Learn how PCA reduces dimensionality in natural language processing by capturing redundancy and correlation in text data, with practical theory and code.

Apply PCA to decorrelate features and enable a gaussian naive Bayes classifier to model p(x|y) and p(y) using Bayes' rule, improving handwritten digit classification.

Provide feedback through the suggestion box to help tailor unsupervised deep learning in Python, sharing your background, course, difficulty, missing explanations, and requests for future topics.

Explore t-SNE theory, a non-linear dimensionality reduction method that preserves local structure by minimizing KL divergence between high- and low-dimensional distributions, with practical limits on speed and memory.

Explore unsupervised deep learning in Python with t-SNE on the donut data, generating 600 points (inner and outer radii, theta) and comparing raw and projected plots.

Apply t-SNE to the MNIST dataset to visualize class separation in an unsupervised setting, compare results to PCA, and note limitations for new data.

Explore autoencoders in unsupervised deep learning, where networks learn to reconstruct inputs from themselves using sigmoid activations, biases, shared weights, and regularization.

Implement an autoencoder class in Theano for unsupervised deep learning in Python, building the network, defining training with momentum, and using squared error or cross-entropy for reconstruction.

Compare greedy layer-wise autoencoder pre-training with pure backpropagation in unsupervised deep learning using Python, showing faster convergence with pre-training and the use of squared error and cross-entropy losses.

Explore a deep autoencoder architecture called the X-wing, with many layers and shared weights to visualize data in a two-center representation. Observe how the center units yield well-separated classes.

Explore unsupervised deep learning in Python with a one-line autoencoder that learns nonlinear latent representations, showing PCA objectives align with autoencoders and the benefits of greedy layer wise pre-training.

Explore how restricted Boltzmann machines use greedy layer-wise pre-training to uncover compact latent representations. See how they relate to auto encoders for removing redundancy and noise in data.

Unpacks the theoretical motivation for restricted Boltzmann machines, linking Boltzmann machines, energy functions, and probabilistic models with visible and hidden units in a statistical mechanics framework.

Explore the intractable nature of computing the normalizing constant for a Bernoulli vector model, highlighting exponential time complexity with a 784-pixel image example and a hyperparameter m set to 100.

Derive neural network equations from energy and probability in restricted Boltzmann machines using Bayes rule, normalizing constants, and Bernoulli hidden units, yielding P(h|v) as the sigmoid of Wv plus b.

Demonstrates training an rbm with maximum likelihood, using the free energy to substitute intractable sums, and applies gradient descent with a positive phase for observed data and a negative phase for others.

Explore greedy layer-wise pretraining with restricted Boltzmann machines to overcome vanishing gradients and build supervised networks by stacking RBMs into hidden representations and adding a logistic regression layer with fine-tuning.

Implement a restricted boltzmann machine in Theano with greedy layer-wise pre-training on mnist for unsupervised learning. Build the rbm object, sample with random streams, and train with a free-energy objective.

Demonstrate the vanishing gradient problem with a hands-on unsupervised deep learning neural network in Python, covering forward passes, backpropagation, momentum updates, and trainable weights and biases across multiple hidden layers.

Apply t-sne and k-means to identify clusters of related words from a tf-idf term-document matrix, visualize them in two dimensions, and inspect the resulting word clusters.

Explore how autoencoders and RBMs learn latent representations to fill in missing data and generate recommendations, using patterns in user ratings and genres to predict preferences.

Cover data preprocessing for movie ratings, converting IDs to zero-based indices and shrinking to active users and movies. Build lookup dictionaries and perform a train-test split, saving results with pickle.

Extend deep learning rbm to a categorical rbm for recommender systems by using a 10-category rating distribution with softmax and handling missing ratings through zero-filled vectors.

Build and train a restricted Boltzmann machine for recommender systems in Python, detailing data loading, RBM class, training loop, free energy objective, and MSE evaluation.

Speed up the RBM recommender in Python by moving one-hot encoding, masking, and mean squared error calculations into TensorFlow operations, eliminating per-batch preprocessing and improving performance.

Explore Theano basics, from symbolic variables and tensors to a computation graph, then build a cost, create shared variables, compute gradients automatically, and train with updates via a train function.

Clarify the appendix and FAQ as optional, supplementary material for this course. Encourage students to post questions in the Q&A to receive answers promptly.