Prior, posterior, likelihood, MAP, ML

Let be a parameter and be the observation or data.

The posterior density is the probability density of given the observation, that is, .

The a priori (or prior) density is the probability density of before any observations are made, that is, .

The likelihood density is the probability of observed data, given the model parameters, that is, .

Using Bayes' rule, it is easy to see that

or, posterior prior likelihood.

When learning an unknown parameter from data, two commonly used estimation methods are as follows.

1. Maximum a posteriori (MAP) estimate

2. Maximum likelihood (ML) estimate

Notice that under a uniform prior, MAP and ML are identical.