Structured Interviews for Pilot Selection: No Incremental Validity

Pitfalls of Ability Research in Aviation Psychology

“…There is a tendency for researchers to engage in the "topological fallacy" (Walters, Miller, & Ree, 1993), that is to interpret a measurement scale or test on the basis of the topology or appearance of the questions. The problem is that the appearance of the questions is not necessarily a true indicator of the construct measured by the scale or test.…”

“…Despite the apparent differences between the IQ test and Space Fortress, Rabbitt et al found a correlation between them of .69 (uncorrected), a value as high as normally found between two different IQ tests. Walters et al (1993) who examined the validity and incremental validity of an experimental structured interview for selecting US Air Force pilot candidates provided another example. Two hundred twenty-three (223) participants completed the Air Force Officer Qualifying Test (AFOQT; , experimental computer-based cognitive and personality tests, and the structured interview.…”

“…The predictive utility of the interview clearly came from its measurement of gThese two studies demonstrated the value of using a reference test. Rabbitt et al (1989) used the AH 4 as a reference test and Walters et al (1993) used the AFOQT as a reference test. Had the researchers not used reference tests they might have concluded that their measures were different from existing constructs.…”

“…In aviation psychology, criterion-based regression-weighting frequently is used (Walters et al, 1993), even though several studies argue for unit or simple weighting. More than three decades ago, Aiken (1966) thought the controversy over the use of simple weights was settled and was surprised to find colleagues arguing for regression-based weights on an intuitive basis.…”

USAF Summer Research Program - 1994 Summer Faculty Research Program Final Reports, Volume 2A, Armstrong Laboratory.

“…Previous studies of fledgling aviators used the completion of training as the criterion of success rather than actual mission readiness or performance in the aircraft cockpit (Carretta, 1989;Carretta & Ree, 1993b;Siem, 1992;Walters et al, 1993). These studies provide limited usefulness because not every pilot successfully transitions from undergraduate pilot training to advanced or upgrade training.…”