Bayesian Computational Analyses with R

The pnorm( ) function is the cumulative density function or CDF. It returns the area below the CDF and to the left up to some point "x" along the horizontal axis, for example, "x = 1."

In probability theory and applications, Bayes's rule relates the odds of event to the odds of event , before (prior to) and after (posterior to) conditioning on another event . The odds on to event is simply the ratio of the probabilities of the two events. The prior odds is the ratio of the unconditional or prior probabilities, the posterior odds is the ratio of conditional or posterior probabilities given the event . The relationship is expressed in terms of the likelihood ratio or Bayes factor, . By definition, this is the ratio of the conditional probabilities of the event given that is the case or that is the case, respectively. The rule simply states: posterior odds equals prior odds times Bayes factor.

In statistics, a likelihood function (often simply the likelihood) is a function of the parameters of a statistical model. Likelihood functions play a key role in statistical inference, especially methods of estimating a parameter from a set of statistics. In informal contexts, "likelihood" is often used as a synonym for "probability." But in statistical usage, a distinction is made depending on the roles of the outcome or parameter. Probability is used when describing a function of the outcome given a fixed parameter value. For example, if a coin is flipped 10 times and it is a fair coin, what is the probability of it landing heads-up every time? Likelihood is used when describing a function of a parameter given an outcome. For example, if a coin is flipped 10 times and it has landed heads-up 10 times, what is the likelihood that the coin is fair?

Discrete priors are in contrast to continuous priors. Discrete priors refers to a set of whole numbers describing the frequency of outcomes of some event, for example, the number of consecutive tosses of "heads" in a series of tests, or samples, of the likelihood of this event. Since cumulative density functions are continuous, one needs to apply 'adjusting functions' to discrete priors to produce continuous posterior distributions.

Beta priors may be used to approximate a continuous CDF distribution for discrete event-based occurrences, such as with the use of a binomial distribution to estimate the number of "success" and "failure" outcomes in the toss of a coin.

The prior predictive distribution, in a Bayesian context, is the distribution of a data point marginalized over its prior distribution.

The use of single parameter models may be exemplified when one is trying to estimate the most likely mean parameter values, or the most likely standard deviation parameter values, but not both (that would be a multi-parameter model).

"Conjugate" models in the Bayesian approach simply mean that the functional form of the density function for both the prior distribution and the posterior distribution are similar, for example, both normally distributed. However "mixtures" refers to Bayesian models where there may be two different, and competing, components to the prior distribution, and one seeks an estimate of which of the two components is more likely, or more tenable.

In probability theory, a probability density function (PDF), or density of a continuous random variable, is a function that describes the relative likelihood for this random variable to take on a given value. For example, the PDF for a normally-distributed random variable takes the shape of the familiar "bell curve."

In the Bayesian approach, multiparameter models are models in which one is attempting to estimate the probability density functions for more than one parameter, for example, both the mean and standard deviation of the target posterior parameters.

In probability theory, the multinomial distribution is a generalization of the binomial distribution. For n independent trials each of which leads to a success for exactly one of k categories, with each category having a given fixed success probability, the multinomial distribution gives the probability of any particular combination of numbers of successes for the various categories.

The binomial distribution is the probability distribution of the number of successes for one of just two categories in n independent Bernoulli trials, with the same probability of success on each trial. In a multinomial distribution, the analog of the Bernoulli distribution is the categorical distribution, where each trial results in exactly one of some fixed finite number k possible outcomes, with probabilities p₁, ..., p_k

An integral is a mathematical object that can be interpreted as an area or a generalization of area. For example, to calculate the area under the "curve" of a continuous function f(x) up to some point "x" along the horizontal axis, one might compute the integral of f(x) at "x." Computing integrals are useful for finding probabilities that are represented as areas under a continuous function "curve" or plot.

In probability theory and statistics, the beta-binomial distribution is a family of discrete probability distributions on a finite support of non-negative integers arising when the probability of success in each of a fixed or known number of Bernoulli trials is either unknown or random. The beta-binomial distribution is the binomial distribution in which the probability of success at each trial is not fixed but random and follows the beta distribution. It is frequently used in Bayesian statistics, empirical Bayes methods and classical statistics as an overdispersed binomial distribution.

In mathematics, rejection sampling is a basic technique used to generate observations from a distribution. It is also commonly called the acceptance-rejection method or "accept-reject algorithm" and is a type of Monte Carlo method. The method works for any distribution in with a density.

In statistics, importance sampling is a general technique for estimating properties of a particular distribution, while only having samples generated from a different distribution than the distribution of interest. It is related to umbrella sampling in computational physics.