Numerical Methods and Optimization in Python

Explain floating point representation using sign bit, exponent, and mantissa, detailing 32-bit single precision and 64-bit double precision, including bias and value ranges.

Explain accuracy versus precision in floating point numbers, compare 32-bit and 64-bit representations, and show Python's 64-bit double delivering about 14-15 digits of accuracy.

Analyze rounding errors in floating point arithmetic, showing how 100 times 0.1 deviates from 10 due to finite precision, and use absolute difference with epsilon to compare values.

Examine performance across Java, C, and Python for numerical computations, highlighting why Python remains popular for science with NumPy and Pandas, and how vectorization and memory locality boost speed.

Explore matrix multiplication and matrix-vector operations, treating matrices as two-dimensional arrays that represent linear maps. See step-by-step computation and real-world applications in computer science, physics, and engineering.

Explore matrix operations, focusing on matrix-matrix multiplication. Treat a matrix as a two-dimensional array of numbers representing linear maps, with rows, columns, and Python's zero-based indexing.

Implement matrix multiplication from scratch in Python by defining a multiply function, initializing two dimensional list C with zeros, and triple looping to compute C[i][j] = sum A[i][k] * B[k][j].

Analyze the running time of naive matrix multiplication with three nested loops, outline divide-and-conquer and block matrix approaches, and compare practical parallelism and Strassen's algorithm with n^2.3 complexity.

Multiply a matrix by a vector in Python with a nested loop, computing row-by-row products and storing results in the result vector.

Compute the inner product by multiplying corresponding vector elements and summing the results to produce a scalar, demonstrated with a loop and with Python zip and sum.

Compare Python lists with NumPy arrays to see why NumPy delivers faster performance through contiguous memory and real arrays. Learn how lists store references versus NumPy's continuous block memory.

Learn to perform matrix operations in Python using NumPy: import NumPy as mp, create multi-dimensional arrays, multiply matrices, compute vector inner products with dot products, and explore vector-matrix operations efficiently.

Learn how Gaussian elimination converts a linear system in matrix form into an upper triangular form with an augmented matrix, then apply back substitution.

Explore Gaussian elimination with elementary row operations to transform a linear system into an upper triangular form and solve for x, y, z through back substitution.

Learn how partial pivoting enhances Gaussian elimination stability and helps avoid ill-conditioned matrices, while complete pivoting offers further stability improvements.

Explore Gaussian elimination for linear systems and identify singular matrices via determinant zero, which means no solution. Understand near-singular matrices and how small changes can cause numerical instability.

Implement gaussian elimination in Python with NumPy to transform matrix A and vector B by zeroing below the pivot using lambda-based row operations.

Implement Gaussian elimination with back substitution to solve linear systems, setting below-diagonal entries to zero, computing unknowns from last to first, and storing results in the b vector.

Use Gaussian elimination on a linear system to determine how to allocate 24,000 between two bonds for a portfolio to earn 930 dollars annually.

Use a prebuilt Gaussian elimination function to solve a linear system for portfolio optimization, building the matrix and vector from bond returns and investments to yield 18,000 and 6,000 allocations.

Explore eigenvalues and eigenvectors, learn to solve det(A-λI)=0 and compute eigenvectors via Gaussian elimination, with a 2x2 example, Python implementation, and applications in engineering, quantum physics, machine learning, and PageRank.

Use NumPy's linear algebra in Python to compute eigenvalues and eigenvectors, with normalized eigenvectors, from a 2x2 matrix and diagonal matrices as examples.

Demonstrate how eigenvectors and eigenvalues enable principal component analysis to reduce high-dimensional image data, improving face recognition and classification with a support vector classifier.

Explore how the PageRank algorithm ranks websites by analyzing the world wide web as a directed graph of hyperlinks and inbound, outbound, and dangling links in search results.

Explore how breadth-first search enables web crawlers to map the web as a directed graph by visiting neighbors layer by layer.

Describe original PageRank formula on a directed web graph, where a page's rank depends on its linking pages. Initialize each page to 1/N and iteratively update, normalizing by outbound links.

Explore the PageRank algorithm through a four-page network, updating ranks via incoming links over outgoing links from a 1/n initialization.

Transform PageRank computation into a transition matrix problem, apply the power method to converge to the equilibrium PageRank vector for four web pages using matrix-vector multiplication.

Compute PageRank values using the steady state approach and the transition matrix, linking eigenvectors to the stationary distribution in the random surfer model.

Analyze how dangling nodes and independent clusters break the PageRank random surfer model and how damping addresses these issues.

Compute PageRank values using the Google matrix, combining the transition matrix with teleportation via the damping factor, and solve for the principal eigenvector whose entries sum to one.

Apply the power method to compute PageRank as the eigenvector of the Google Matrix with eigenvalue one, using sparse, approximate computation on enormous graphs.

Explore interpolation in two dimensions, contrast it with regression, and learn to minimize a squared error cost function using polynomial models and a matrix formulation for exact data points.

Define the cost function, compute its partial derivatives, and solve for interpolation parameters. Construct the interpolation matrix, apply Gaussian elimination, and obtain quadratic coefficients that fit all data points.

Implement Lagrange interpolation by building a matrix of powers of x from data points and solving for polynomial coefficients with Gaussian elimination in NumPy, for orders from linear to cubic.

Implement lagrange interpolation by solving a matrix equation with gaussian elimination to obtain the interpolation polynomial parameters and plot the results.

Explore the applications of interpolation to generate new data points for time series and machine learning, approximate values like temperature or stock prices, and fill in image pixels during resizing.

Explore iterative root finding algorithms to locate zeros of a function from an initial point with bracketing bounds. Learn derivative-based and binary-search–like methods to find maxima, minima, or solve f(x)=C.

Discover how the bisection method finds a root by using two opposite-signed points, evaluating the midpoint, and slicing the interval until precision.

Implement the bisection method from scratch, defining x negative, x positive, and an epsilon. Iterate to compute the middle point and update bounds to approach the root of the function.

Introduce the Newton method as an iterative root-finding approach using x_{k+1}=x_k - f(x_k)/f'(x_k), discuss derivative reliance, convergence thresholds, and switching to the by section method when needed.

Implement the Newton method for f(x)=x^2-2, using x_{n+1}=x_n - f(x_n)/f'(x_n) starting at 1 to approximate sqrt(2); it is fast but requires the derivative and can be unstable near extrema.