Finding Outpoints in Noisy Binary Sequences — A Revised Empirical Evaluation

Suboptimal behavior of Bayes and MDL in classification under misspecification

“…sample sizes. However, according to Viswanathan et al (1999), this is caused by the coding method used to encode hypotheses. This method does not take into account the precision of parameters involved (whereas taking the precision into account is a crucial aspect of MDL!).…”

“…This method does not take into account the precision of parameters involved (whereas taking the precision into account is a crucial aspect of MDL!). In the paper (Viswanathan et al, 1999), a different coding scheme is proposed. With this coding scheme, MML (an inference method that is related to MDL, see below) apparently behaves quite well on the classification problem studied by Kearns et al In contrast to Kearns' example, in our case (a) there is no straightforward way to improve the coding scheme; (b) MDL fails even on an infinite sample.…”

MML, Hybrid Bayesian Network Graphical Models, Statistical Consistency, Invariance and Uniqueness

“…2;Wallace, 2005, sec. 7.3;Viswanathan et al, 1999;Wallace and Patrick, 1993;Comley and Dowe, 2005, secs. 11.3 and 11.4.3;Needham and Dowe, 2001;Wallace, 2005, sec.…”

“…While these two notions are different, it should be pointed out that, insofar as reliable MML coding schemes lead to good inferences and less reliable coding schemes lead to less reliable inferences [Quinlan and Rivest, 1989;Patrick, 1993, Kearns et al, 1997;Viswanathan et al, 1999;Wallace, 2005, sec. 7.3;Murphy and Pazzani, 1994;Needham and Dowe, 2001;Wallace and Dowe, 1999a, secs.…”

“…People should take as much care as they are able to here - [Comley and Dowe, 2005, sec. 11.4.3] gives plenty of examples of poor MDL-like coding schemes whose performances vastly improved when they were re-visited using MML [Wallace and Patrick, 1991;, Viswanathan et al, 1999Needham and Dowe, 2001]. (See also sec.…”