R Programming/Maximum Likelihood

Introduction
Maximum likelihood estimation is just an optimization problem. You have to write down your log likelihood function and use some optimization technique. Sometimes you also need to write your score (the first derivative of the log likelihood) and or the hessian (the second derivative of the log likelihood).

One dimension
If there is only one parameter, we can optimize the log likelihood using.

Example with a type 1 Pareto distribution
We provide an example with a type 1 Pareto distribution. Note that in this example we treat the minimum as known and do not estimate it. Therefore this is a one-dimensional problem.

We use the  (actuar) function to generate a random vector from a type 1 Pareto distribution with shape equal to 1 and minimum value equal to 500. We use the  (actuar) function with option   to write the log likelihood. Then we just need to use  with. We provide a minimum and a maximum value for the parameter with the  option.

Multiple dimension

 * fitdistr (MASS package) fits univariate distributions by maximum likelihood. It is a wrapper for.
 * If you need to program yourself your maximum likelihood estimator (MLE) you have to use a built-in optimizer such as nlm, optim. R also includes the following optimizers :
 * mle in the stats4 package
 * The maxLik package

Example with a logistic distribution
For instance, we draw from a logistic distribution and we estimate the parameters using.

Example with a Cauchy distribution
For instance, we can write a simple maximum likelihood estimator for a Cauchy distribution using the nlm optimizer. We first draw a vector x from a Cauchy distribution. Then we define the log likelihood function and then we optimize using the nlm function. Note that nlm is minimizer and not a maximizer.

Example with a beta distribution
Here is an other example with the Beta distribution and the optim function.

Likelihood Ratio Test

 * lrtest in the lmtest package.

Some Specific cases

 * <tt>gum.fit</tt> (ismev package) provides MLE for a Gumbel distributon

Resources

 * Charles Geyer : Maximum Likelihood in R (pdf, 9 pages)
 * Marco Steenbergen Maximum Likelihood Programming in R (pdf, 7 pages)