Stochastic Processes, Markov Chains and Markov Jumps

Explore the Markov chain as a discrete-time stochastic process with a discrete state space and the Markov property, showing transitions between healthy, ill, and dead.

Learn transition probabilities in a Markov chain, denoted p_ij^m(n). Understand transitions between states like healthy, corona, and dead, and crude estimation via N_ij divided by N_i.

Explain the Chapman–Kolmogorov equation using a Markov chain, showing how the probability of moving from healthy to death over a period equals the sum of two-step paths via intermediate states.

Convert a markov chain diagram into a transition matrix P with states healthy, corona, and dead. Understand one-step probabilities, row sums equal to one, and time-homogeneous simplifications.

Explore a transition matrix in a Markov chain, calculating one-, two-, and three-period probabilities of dying from healthy. Implement the calculations in R using a 3x3 matrix and matrix multiplication.

Learn how to restore the Markov property by modifying states: introduce a recovered state to distinguish past infection history, yielding a transition diagram and matrix with an absorbing dead state.

Stationary probability distributions describe the long-run distribution of people across states in a Markov chain, using the stationary vector pi and the transition matrix to show stable proportions.

Explore irreducibility in Markov chains by identifying whether a chain can reach every state, using transition diagrams and reduced chains to determine long-term stationary distributions.

Explore periodicity in a simple Markov chain, showing state one's period two and that all states in an irreducible chain share this period, with convergence to a unique stationary distribution.

Install and set up R and RStudio following the actuarial guide, including downloading R, XQuartz, and essential packages like ggplot2, graphics, and stats to begin data work.

Learn the basics of using R for actuaries, part 1, including installing packages, vectors, sequences, repeats, indexing, for and while loops, if statements, functions, and core statistics.

Master practical R basics for actuaries, including vectors, matrices, indexing, transposition, and matrix multiplication. Explore data frames, lists, dplyr, linear and generalized linear models, t-tests, distributions, histograms, and plotting.

Explains how to interpret a complex stochastic processes exam question on Poisson arrivals, pod and taxi dynamics, and how to build the transition matrix and compute taxi usage.

Compute probabilities for 0–3 and 4 or more passengers in a two-minute period using a Poisson distribution with lambda 2, and show the R code to obtain and sum them.

Build a transition matrix for a Poisson-driven passenger queue with pods and taxis using a Markov chain, analyzing states 0 to 4+ and the associated probabilities.

Explore questions 3 and 4 by adjusting the Poisson lambda from 2 to 1.75; estimate hourly taxi demand using the four-or-more passenger probability times 30 pods, with R.

Create a labeled passenger matrix in R. Run a for loop to approximate the markov chain steady state and extract the four-or-more passenger probability to estimate taxis per hour.

Compute the stationary distribution of a five-state Markov chain using a passenger matrix and R code, then interpret the probability of four or more passengers at arrival.

Compare part four to part three to show price frequency lowers taxi frequency, with 17.84% to 12.78% and 5.36 to 4.38 taxis, while cost data govern pods versus taxi choice.

Explore the continuous-time, discrete-state Markov jump process and its rates, with forward and backward differential equations, illustrated by a two-state alive or death example.

Define and contrast transitional probabilities and survival probabilities, explaining how a life at age x transitions to death by age x+T or remains alive to x+T.

Explore Kolmogorov's forward differential equation for Markov jump processes, using the Markov property and small-h mortality approximations with constant mortality for t<1 year.

Learn the math behind Kolmogorov forward differential equation by deriving the relation between survival probability, transition probabilities, and the force of mortality in a Markov model.

Learn to solve Kolmogorov's forward differential equation by using the log-derivative trick, integrate over time, and apply constant mortality to derive the survival function and force of mortality.

Compare Markov chains and Markov jump processes, contrasting discrete-time transition probabilities with continuous-time transition rates, and explain generator matrices, absorbing states, and diagram representations.

Explore Kolmogorov's backwards differential equation, linking the derivative of the transitional matrix to the generator, with applications to healthy to corona and survival probabilities in Markov jump processes.