Math 0-1: Calculus for Data Science & Machine Learning

Review practical function concepts for this course, including real numbers as inputs, function notation, variable letters, linear and polynomial forms, root, exponential, logarithmic, trig, hyperbolic functions, and base changes.

Use Python as a calculator to review functions by plotting linear, quadratic, exponential, log, sine, cosine, tangent, and sigmoid curves, and compare growth and asymptotes in data science.

Explore the concept of limits, distinguishing functions from relations, and how limits define derivatives and differentiation rules, with intuitive examples like holes and sine x over x.

Explore the precise epsilon-delta definition of the limit, including delta, and how for every epsilon there exists a delta ensuring |f(x)-l|<epsilon as x approaches a, noting this is optional.

Learn how limit laws let you evaluate limits, assuming they exist, using addition, subtraction, constant multiples, products, quotients (when the denominator limit is nonzero), powers, and roots to build intuition.

Explore limits involving infinity and asymptotes through concrete examples. See 1/x as x→∞ approach 0, x^2 → ∞, and tan x near pi/2 where the limit does not exist.

Explore indeterminate forms in limits, including zero over zero and infinity over infinity, with intuitive examples and notes on how limits can resolve them.

Explore how to compute limits in Python using NumPy and Matplotlib, including 1/x as x grows and the convergence to zero. See how (1+1/x)^x approaches e amid precision limits.

Visualize limits with plotting in python using numpy and matplotlib, exploring log x over x as x grows, ratios, and sin x over x and (cos x minus 1)/x limits.

Understand the limit notation and that the limit may differ from f(a). Explore one‑sided limits, infinities, asymptotes, indeterminate forms, and intuitive plotting to evaluate them.

Derive the slope of tangent lines via the limit definition, relate differentiability to continuity, and identify non-differentiable cases such as cusps, jumps, and vertical tangents, with ReLU as an example.

Explore how the limit of (f(x)-f(a))/(x-a) yields the tangent line, and compare one-sided versus two-sided difference quotients, with applications to derivative checking and black-box functions in neural networks.

Practice using the limit definition of the derivative to derive f'(x) for f(x) = x^2, showing cancellation and the limit as h approaches zero, yielding f'(x)=2x.

Apply the limit definition of the derivative to y = x^3. Expand (x + h)^3, then evaluate the limit to obtain dy/dx = 3x^2.

Explore the derivative of one over x using first principles, confirming that f'(x) = -1/x^2 and expressing it as -x^{-2}.

Using the limit definition and rationalizing the numerator, derive the derivative of f(x)=sqrt(x) as f'(x)=1/(2 sqrt(x)), and explore its behavior near zero and infinity.

Explore alternative derivative notations, including df/dx, f′(x), dy/dx with a vertical bar for evaluation, and the capital d notation, plus higher-order derivatives f^(n)(x) and ẏ, ÿ.

Learn derivative checking by comparing finite difference estimates with true derivatives for x squared, x cubed, 1/x, and sqrt(x) using small h in Python.

Define derivatives via limits, showing slope as a function, use the two sided limit for derivative checking, and cover notations like f' x and dy/dx, plus higher orders; neural networks.

Derive the power rule for d/dx x^n as n x^{n-1} using a difference-quotient method and the identity x^n - a^n = (x - a)(x^{n-1} + x^{n-2} a + ... + a^{n-1}).

Explains constant multiple rule and the sum and difference rules for derivatives, derived from limit laws, and shows how to differentiate constants, polynomials, and simple functions like x and 5x^2.

Derive the exponential rule for B^x using first principles and the limit definition. Recognize that f'(x) = (ln B) B^x, and that e^x differentiates to itself.

An alternative derivation of the exponential rule uses the limit definition of e, then derives d/dx e^x from first principles, and extends to general bases via natural log.

Learn the chain rule for composite functions, derive y equals f of g of x by multiplying derivatives, and see its application to neural networks, deep learning, and auto differentiation.

Explore chain rule exercises, including differentiating (1+x^2)^20, e^{-x^2}, and the sigmoid 1/(1+e^{-x}), using u-substitution and expressing y' in terms of y.

learn the product rule and prove d/dx [f(x) g(x)] = f(x) g'(x) + f'(x) g(x). derive the quotient rule via limits and connect it to the chain rule.

Practice applying the product and quotient rules in calculus for data science, deriving derivatives for two-function and three-function products using f and g.

Explore implicit differentiation for derivatives of relations, such as the circle x^2 + y^2 = 1. Use the chain rule to obtain dy/dx without isolating y, yielding -x/y.

Apply implicit differentiation to log base b of x by exponentiating to b^y = x, then dy/dx = 1/(x ln b) for x>0, noting the domain and base e simplification.

Explore applications of implicit differentiation and prove the power rule, product rule, and quotient rule using logarithmic differentiation and derivative of absolute value, with step-by-step derivations.

Use logarithmic differentiation to simplify derivatives by taking logs of both sides, demonstrated on examples like y = x^(3/4) sqrt(x^2+1)/(3x+2)^5, x^x, and y = f(x)^{g(x)} that involve multiple rules.

Explore derivatives of hyperbolic functions, focusing on tanh x, derive y' = 1 − y^2 (or sech^2 x), and show how the derivative connects to identities for deep learning.

Derive the derivative of a Gaussian with respect to the variance, using logarithmic differentiation on y = (1/√V) e^{-1/2 x^2 / V}, treating V as the input.

Study entropy and log loss through hands-on differentiation in machine learning. Differentiate with respect to x and p, applying product and chain rules and constants t and w.

Derive the derivatives of sine, cosine, and tangent using key limits and identities, including sin x over x equals 1 and (cos x minus 1)/x equals 0.

Explore core derivative rules including power, constant multiple, addition, and quotient via product and chain rules; learn exponential, logarithmic, trigonometric, and inverse derivatives, plus implicit and logarithmic differentiation.

Explore how differentiation identifies minima and maxima, guiding machine learning aims by locating zero-derivative points. Explain global versus local minima and zero-derivative cases.

Define local maxima and minima using epsilon neighborhoods to distinguish them from global extrema, illustrated with intuitive neighborhood visuals. Discuss constrained optimization and note applications in control theory and finance.

Use the second derivative test to classify critical points: if f''(x) > 0, the point is a concave up minimum; if f''(x) < 0, it's a concave down maximum.

Learn to locate minimums and maximums using derivatives and first/second derivative tests, with x^2, x^3, and x^4 examples for calculus in data science and machine learning.

Analyze entropy for the bernoulli by finding p where y prime is zero, identify p equals one half, and confirm a global minimum with the second derivative test.

Practice exercise finds the Gaussian mean by maximizing y = exp(-1/2 (x-μ)^2) with respect to μ, showing μ* = x through log trick and a second-derivative check for a maximum.

Explore optimizing a Gaussian variant by taking the log, derive the critical point V* = X^2, and confirm a maximum via the second derivative test.

Explore L'Hôpital's rule as a derivative-based tool for evaluating indeterminate limits, with examples including the exponential limit and sin x over x, and cautions about circular logic.

Apply Newton's method to find zeros of a function by using tangent lines, updating x_k to x_{k+1} = x_k - f(x_k)/f'(x_k), and extend to optimization by applying it to f'.

Explore Newton's method in Python to find zeros of a quadratic using numpy and matplotlib, show how starting points influence which zero is found, and verify with the quadratic formula.

Explore applications of differentiation, including locating stationary points with the first and second derivative tests, and using Lafitau's rule for limits and Newton's method for zeros and stationary points.

Introduce integrals as antiderivatives, showing how calculus underpins probability and statistics, Bayesian machine learning, and Markov chain Monte Carlo, with emphasis on understanding concepts over mastering techniques.

Learn to estimate the area under a curve with rectangles and delta x, then make it exact using the limit and the definite integral from a to b.

Explore the two parts of the fundamental theorem of calculus, linking differential and integral calculus. Learn how the antiderivative F satisfies F'(x)=f(x) and relates to the area under a curve.

Part two of fundamental theorem of calculus shows area under the curve from a to b equals antiderivative at b minus at a, replacing the sum with a subtraction.

Explore the fundamental theorem of calculus, focusing on the first part: the integral as a limit via the Riemann sum (area under the curve) equals the derivative of its antiderivative.

Learn how definite integrals calculate the area under f(x) from A to B by evaluating the antiderivative at B and at A, while indefinite integrals yield F(x) plus C.

Practice definite integrals with simple exercises on area, constants outside the integral, and symmetry of odd and even functions, including lines and quadratics.

Master indefinite integrals by applying the power rule, memorizing the constants of integration, and using substitution to simplify challenging cases, while verifying results by differentiation.

Explore improper integrals, focusing on convergence and divergence with examples like ∫_0^∞ e^{-x} dx = 1 and ∫_1^∞ 1/x dx diverges, relating to probability and statistics for machine learning.

Learn numerical integration in Python by approximating areas under curves with rectangle sums, using numpy linspace to generate x-points and midpoint heights, and compare to exact results.

Explore the fundamental theorem of calculus in two parts, linking antiderivatives to areas. Learn to compute definite, indefinite, improper, and numerical integrals.

Describe functions of multiple variables as vectors, with examples such as laptop temperature T(x,y), the two-dimensional bell curve f(x,y), and vector-valued inputs like velocity and gravity forces via Newton's law.

Master partial differentiation for multivariable functions by differentiating with respect to x or y while keeping other variables constant, using standard notations and interpreting results as tangent slopes on planes.

Explore gradients as the vector of partial derivatives for multivariable functions, reveal the direction of steepest ascent, and apply gradient ascent or descent in data science and machine learning.

Learn how the jacobian forms a matrix of partial derivatives for vector outputs and how the hessian collects second derivatives, illustrated with xy, x^2+y^2, and f=[x,y,z]/(x+y+z).

Explore the chain rule in multiple dimensions by deriving the differential from the first-order Taylor expansion, and apply it to time-dependent functions, gradients, and neural networks.

Explore why the gradient points in the direction of the steepest ascent and how gradient descent uses this direction for efficient, iterative optimization in data science.

Demonstrates that the gradient points to the direction of steepest descent for the function f(x,y)=x^2+y^2, using unit vectors and random directions to compare outcomes.

Master constrained optimization using the Lagrangian to relate gradients of f and g through a multiplier lambda. Solve examples like maximizing x+y subject to x^2+y^2=1 and interpret with contour plots.

Explore why F's gradient is parallel to constraint gradients on level curves, via directional derivative. Learn to handle multiple constraints by formulating a Lagrangian and solving for zero gradient.

Explore partial differentiation, gradients, jacobians, and differentials in multiple dimensions; apply the chain rule, directional derivatives, and gradient descent and ascent in unconstrained and constrained optimization with Lagrange multipliers.

Explore the appendix and FAQ as optional, supplementary sections that provide context and answers, and use the Q&A to ensure you have zero unanswered questions.

This lecture explains that installation lectures are generic, scalable guidelines focused on learning principles over syntax, using examples like pip installation and libraries such as Theano, CNTK, and OpenAI gym.

Learn to install and manage data science libraries on Windows with Anaconda, including Python 3/2, conda and pip, and verify TensorFlow, PyTorch, Keras, CNTK, OpenAI Gym, and more.

Set up a cross-platform data science environment by installing numpy, scipy, matplotlib, pandas, ipython, theano, and tensorflow; learn to use VirtualBox with Ubuntu to work around Windows limitations.

Highlights common code access mistakes: notebooks are not on GitHub, you must be logged in to use the code link, and VPN use can block signup and verification emails.

Download the course repository from GitHub via git clone or the download button, and learn essential git basics and coding exercises.