Unsupervised Deep Learning in Python

Pca stands for principal components analysis and helps transform high-dimensional data into useful features for downstream algorithms, and also enables visualization in two or three dimensions.

Rotate data with a pca orthonormal transformation to preserve vector magnitudes and angles, effectively rotating the coordinates while keeping lengths intact.

Explore unsupervised learning with PCA on the MNIST dataset by loading data, fitting a PCA model, and visualizing two-dimensional projections, explained variance, and cumulative variance to select principal components.

Learn how PCA reduces dimensionality in natural language processing by capturing redundancy and correlation in text data, with practical theory and code.

Learn how the PCA objective function minimizes squared reconstruction error by transforming X to Z with first k eigenvectors of X's covariance and reconstructing X via Q_k^T, using Frobenius norm.

Apply PCA to decorrelate features and enable a gaussian naive Bayes classifier to model p(x|y) and p(y) using Bayes' rule, improving handwritten digit classification.

Provide feedback through the suggestion box to help tailor unsupervised deep learning in Python, sharing your background, course, difficulty, missing explanations, and requests for future topics.

Explore t-SNE theory, a non-linear dimensionality reduction method that preserves local structure by minimizing KL divergence between high- and low-dimensional distributions, with practical limits on speed and memory.

See how t-SNE visualizes high-dimensional unsupervised data by reducing to two dimensions, preserving relative distances, with color-coded gaussian clouds arranged at cube corners.

Explore unsupervised deep learning in Python with t-SNE on the donut data, generating 600 points (inner and outer radii, theta) and comparing raw and projected plots.

Apply t-sne to an xor-like unsupervised dataset in Python, visualize original four-box points, and compare with the transformed projection to show how labels do not guide the embedding.

Apply t-SNE to the MNIST dataset to visualize class separation in an unsupervised setting, compare results to PCA, and note limitations for new data.

Explore autoencoders in unsupervised deep learning, where networks learn to reconstruct inputs from themselves using sigmoid activations, biases, shared weights, and regularization.

Learn denoising autoencoders as a regularization method in unsupervised deep learning in Python by reconstructing original inputs from noisy data through gaussian noise or zeroed features to improve generalization.

Implement an autoencoder class in Theano for unsupervised deep learning in Python, building the network, defining training with momentum, and using squared error or cross-entropy for reconstruction.

Build a deep neural network class for unsupervised learning, implementing a deep autoencoder with hidden layers in Theano, and cover supervised training with pre-training options and softmax classification.

Explore unsupervised deep learning in Python by building an auto encoder and a deep neural network with greedy layerwise pre-training in TensorFlow, then finalize with supervised training.

Compare greedy layer-wise autoencoder pre-training with pure backpropagation in unsupervised deep learning using Python, showing faster convergence with pre-training and the use of squared error and cross-entropy losses.

Explore cross entropy and Khail divergence in unsupervised deep learning, linking cross entropy to entropy and Khail divergence through information theory concepts and autoencoder training.

Explore a deep autoencoder architecture called the X-wing, with many layers and shared weights to visualize data in a two-center representation. Observe how the center units yield well-separated classes.

Explore unsupervised deep learning in Python with a one-line autoencoder that learns nonlinear latent representations, showing PCA objectives align with autoencoders and the benefits of greedy layer wise pre-training.

Explore how restricted Boltzmann machines use greedy layer-wise pre-training to uncover compact latent representations. See how they relate to auto encoders for removing redundancy and noise in data.

Unpacks the theoretical motivation for restricted Boltzmann machines, linking Boltzmann machines, energy functions, and probabilistic models with visible and hidden units in a statistical mechanics framework.

Explore the intractable nature of computing the normalizing constant for a Bernoulli vector model, highlighting exponential time complexity with a 784-pixel image example and a hyperparameter m set to 100.

Derive neural network equations from energy and probability in restricted Boltzmann machines using Bayes rule, normalizing constants, and Bernoulli hidden units, yielding P(h|v) as the sigmoid of Wv plus b.

Demonstrates training an rbm with maximum likelihood, using the free energy to substitute intractable sums, and applies gradient descent with a positive phase for observed data and a negative phase for others.

Learn to train an rbm using contrastive divergence with one-step Gibbs sampling, approximating the gradient via the free energy difference and leveraging automatic differentiation for optimization.

Learn to compute the free energy for Bernoulli RBMs without intractable sums. Derive a practical, implementable expression that enables training by simplifying the energy and log product terms.

Explore greedy layer-wise pretraining with restricted Boltzmann machines to overcome vanishing gradients and build supervised networks by stacking RBMs into hidden representations and adding a logistic regression layer with fine-tuning.

Implement a restricted boltzmann machine in Theano with greedy layer-wise pre-training on mnist for unsupervised learning. Build the rbm object, sample with random streams, and train with a free-energy objective.

Demonstrate the vanishing gradient problem with a hands-on unsupervised deep learning neural network in Python, covering forward passes, backpropagation, momentum updates, and trainable weights and biases across multiple hidden layers.

Apply latent semantic analysis to a corpus of book titles, preprocess with tokenization and stopword removal, build a binary document-term representation, and visualize semantic clusters in two-dimensional space.

Apply t-sne and k-means to identify clusters of related words from a tf-idf term-document matrix, visualize them in two dimensions, and inspect the resulting word clusters.

Apply autoencoders and RBMs to a recommender system built from sparse user-item ratings. Explore Movieland data and unsupervised learning to understand how these models work and why they matter.

Explore how autoencoders and RBMs learn latent representations to fill in missing data and generate recommendations, using patterns in user ratings and genres to predict preferences.

Master data preparation for unsupervised deep learning in Python, preprocess user and movie ids, shrink data with top users and movies, and build sparse train-test matrices for autoencoders and RBM.

Cover data preprocessing for movie ratings, converting IDs to zero-based indices and shrinking to active users and movies. Build lookup dictionaries and perform a train-test split, saving results with pickle.

Explore AutoRec in Python by building a simple autoencoder for recommender systems, featuring data preprocessing, CSR sparse matrices, train–test split, masking, dropout, and a custom loss in keras.

Extend deep learning rbm to a categorical rbm for recommender systems by using a 10-category rating distribution with softmax and handling missing ratings through zero-filled vectors.

Explore implementing a recommender system with a restricted Boltzmann machine in Python, focusing on custom one-hot encoding, scaling fractional ratings, missing rating masks, and softmax-based rating prediction.

Build and train a restricted Boltzmann machine for recommender systems in Python, detailing data loading, RBM class, training loop, free energy objective, and MSE evaluation.

Speed up the RBM recommender in Python by moving one-hot encoding, masking, and mean squared error calculations into TensorFlow operations, eliminating per-batch preprocessing and improving performance.

Explore Theano basics, from symbolic variables and tensors to a computation graph, then build a cost, create shared variables, compute gradients automatically, and train with updates via a train function.

learn to implement a Theano-based neural network in Python, covering data setup, softmax, a cost function with negative log-likelihood and regularization, and train and prediction functions.

This lecture reviews TensorFlow basics, introducing placeholders, variables, and matrix multiplication, and demonstrates training a simple model with a gradient descent optimizer through a session and feed_dict.

This lecture reviews TensorFlow basics by building a neural network with two hidden layers, defining weights, biases, and placeholders, and training with softmax cross-entropy with logits to evaluate error rates.

Clarify the appendix and FAQ as optional, supplementary material for this course. Encourage students to post questions in the Q&A to receive answers promptly.