## Saturday, January 21, 2012

### Prior, posterior, likelihood, MAP, ML

Let $\theta$ be a parameter and $X$ be the observation or data.

The posterior density is the probability density of $\theta$ given the observation, that is, $p(\theta | X)$.

The a priori (or prior) density is the probability density of $\theta$ before any observations are made, that is, $p(\theta)$.

The likelihood density is the probability of observed data, given the model parameters, that is, $p(X | \theta)$.

Using Bayes' rule, it is easy to see that

or, posterior $\propto$ prior $\times$ likelihood.

When learning an unknown parameter from data, two commonly used estimation methods are as follows.

1. Maximum a posteriori (MAP) estimate

2. Maximum likelihood (ML) estimate

Notice that under a uniform prior, MAP and ML are identical.