No abstract

The Ada Conformity Assessment Test Suite (ACATS) includes both positive and negative tests. The negative tests have intentional errors that a compiler is intended to diagnose with a compile-time error message. Interestingly, the negative tests also include numerous "OK" lines that the compiler must not reject. But the absence of an error message on the "OK" lines is not always a convincing demonstration that the "OK" lines were correctly compiled, since the negative tests are not executable. By removing the intentional errors from a copy of each negative test, we are able to compile and run the resulting modified tests, demonstrating more convincingly that the "OK" lines are correctly compiled.

The Ada Conformity Assessment Test Suite (ACATS) includes both positive and negative tests. The negative tests have intentional errors that a compiler is intended to diagnose with a compile-time error message. Interestingly, the negative tests also include numerous "OK" lines that the compiler must not reject. But the absence of an error message on the "OK" lines is not always a convincing demonstration that the "OK" lines were correctly compiled, since the negative tests are not executable. By removing the intentional errors from a copy of each negative test, we are able to compile and run the resulting modified tests, demonstrating more convincingly that the "OK" lines are correctly compiled.

A well-known issue with compiler conformance testing is that the tested environment may differ from the end user's environment, in ways that defy analysis. Possible differences include the host or target computer instruction set, the host or target computer operating system version, version differences in various components of the compilation system, and differences in compilation switch settings. Most of these differences can be eliminated by retesting in the end-user's actual environment. However, if the end user's environment includes compilation switches that suppress some or all of Ada's run-time checks, which we believe to be quite common, it is not currently feasible to re-run ACATS testing in that mode. That is because many ACATS tests rely on run-time checking, and those tests are not segregated or otherwise identified. We propose to remedy this difficulty by identifying such tests, so that the remaining tests can all be run and expected to pass with compilation flags that suppress some or all checks.

A well-known issue with compiler conformance testing is that the tested environment may differ from the end user's environment, in ways that defy analysis. Possible differences include the host or target computer instruction set, the host or target computer operating system version, version differences in various components of the compilation system, and differences in compilation switch settings. Most of these differences can be eliminated by retesting in the end-user's actual environment. However, if the end user's environment includes compilation switches that suppress some or all of Ada's run-time checks, which we believe to be quite common, it is not currently feasible to re-run ACATS testing in that mode. That is because many ACATS tests rely on run-time checking, and those tests are not segregated or otherwise identified. We propose to remedy this difficulty by identifying such tests, so that the remaining tests can all be run and expected to pass with compilation flags that suppress some or all checks.