Maximum Likelihood: An Introduction

Abstract

Maximnm likelihood estimates are reported to be best under all circumstances. Yet there are numerous simple examples where they plainly misbehave. One gives some eranmples for problems that had not been invented for the purpose of annoying ms,aximunm likelihood fans. Another example, imitated from B'hadu'r, has been specially created with just such a purpose in mind. Next, we present a list of principles leading to the construction of good estimates. The main principle says that one should not believe in principles but study each problem for its own sake.One of the most widely used methods of statistical estimation is that of maximum likelihood. Opinions on who was the first to propose the method differ. However Fisher is usually credited with the invention of the name 'maximum likelihood', with a major effort intended to spread its use and with the derivation of the optimality properties of the resulting estimates.Qualms about the general validity of the optimality properties have been expressed occasionally. However as late as 1970 L.J. Savage could imply in his 'Fisher lecture' that the difficulties arising in some examples would have rightly been considered 'mathematical caviling' by R.A. Fisher.Of course nobody has been able to prove that maximum likelihood estimates are 'best' under all circumstances. The lack of any such proof is not sufficient by itself to invalidate Fisher's claims. It might simply mean that we have not yet translated into mathematics the basic principles which underlied Fisher's intuition.The present author has, unwittingly, contributed to the confusion by writing two papers which have been interpreted by some as attempts to substantiate Fisher's claims.To clarify the situation we present a few known facts which should be kept in mind as one proceeds along through the various proofs of consistency, asymptotic normality or asymptotic optimality of maximum likelihood estimates.The examples given here deal mostly with the case of independent identically distributed observations. They are intended to show that maximum likelihood does possess disquieting features which rule out the possibility of existence of undiscovered underlying principles which could be used to justify it. One of the very gross forms of misbehavior can be stated as follows.Maximum likelihood estimates computed with all the information available may turn out to be inconsistent. Throwing away a substantial part of the information may render them consistent.The examples show that, in spite of all its presumed virtues, the maximum likelihood procedure cannot be universally recommended. This does not mean that we advocate L. LE CAM some other principle instead, although we give a few guidelines in ? 6.

Statistical models: Conventional, penalized and hierarchical likelihood

“…Some cases of unsatisfactory behavior of the MLE are reported for instance in [30]. The properties of the MLE may not be satisfactory when the number of parameters is too large, and especially when it increases with n as in an example given by Neymann and Scott [36].…”

Обобщенные Гиперболические Распределения Как Предельные Для Случайных Сумм

“…Согласно расхожему мнению, столь высокая адекватность обобщенных гиперболических мо-делей может быть формально объяснена большим числом настраивае-мых параметров, позволяющим подогнать какую угодно модель к ка-ким угодно данным. Среди статистиков хорошо известно высказывание Ж. Бертрана «Give me four parameters and I shall describe an elephant; with five, it will wave its trunk» (цитируется по статье Л. Ле Кама [17]). Это обстоятельство, конечно же, играет свою роль, однако на самом…”

On the Standard Asymptotic Confidence Ellipsoids of Wald

“…Besides, there is no lack of examples where the likelihood ratio method suffers from major difficulties. See for instance Lehmann (1986) page 342 or Le Cam (1979).…”