Automatic generation of useful syntax error messages

Abstract

Many current compilers produce in some situations wrong error messages that mislead the user and harm his confidence in the system. It is demonstrated that a reliable and efficient syntax error handling system may be produced automatically by a compiler generator from the BNF specification of the language, and without any effort by the language implementor. This result is achieved in three ways: Some errors may not be diagnosed without knowledge of the intentions of the programmer. Some compilers employ a sophisticated analysis that attempts to capture these intentions, but which is not always successful. Such an elaborate analysis is not employed here, and instead a list of all the legal corrections is displayed, so that the programmer may readily select the right one. The recovery symbols are selected by a ‘careful’ algorithm resulting in a high probability for correct error recovery. The ‘honest’ error messages show also the parts of the code which could not be analysed correctly because of errors, and where more errors may exist. Efficiency is achieved by computing the recovery sets once and for all at compiler generation time, so that a fast error recovery at program compilation time is achieved. Experiments with erroneous programs suggest that the method compares well with the best compilers that we have seen, and is specially good at the avoidance of wrong error messages.

How Developers Visualize Compiler Messages: A Foundational Approach to Notification Construction

“…Jeffrey created a tool called Merr that overrides the error handler of the LR parser generator of a compiler to automatically provide more useful syntax error messages [9], and Kantorowitz and Laor likewise propose modifications to the parser generator [10]. While these tools apply to text error messages, they illustrate that tools can improve error messages when they can interact with compiler internals.…”

On Compiler Error Messages: What TheySayand What TheyMean

“…Actually, compiler vendors do receive user complaints on error diagnostics [14]. Low usability in the programming environments may severely affect programmers' productivity [37] and their overall user experience [13]. On the other hand, moving from novice to expert is only a slow gradual process, which can be hindered by the difficulties of error messages (among other reasons).…”

“…An error recovery method which can be generated automatically given the grammar of the language is given in [13]. Despite being automatic, their experiments on 600 Pascal programs written by students suggest the method performs satisfactorily in comparison with the best system the authors knew.…”

“…Moreover, messages should provide programmers with guidance in what they could do to fix the error. Technically speaking, it might be hard or impossible to determine with precision what the programmer's intention was [13], but alternatives, possibly based on user profile, could be suggested. As an example, all g++ error messages analyzed in Appendix A fail to provide useful suggestions or even make hard to notice where the problem actually is.…”

“…While there seems not to be any formal study on why commercial compilers have neglected the area of error diagnostics, suggested ideas include the fact that higher priority has been paid to other product features such as compilation speed or the speed of the resulting executable program [11,12], or that the developer team usually has tight schedules or does not have enough experience or the required skills to work on it [13]. In 1999, Alexandrescu, a C++ expert and developer, wrote an open letter addressed to C++ compiler vendors and the C++ community with a short proposal "to make diagnostic messages generated by C++ compilers easier to read and understand in the presence of templates" [14].…”