Reliability and Interrater Agreement: Development and Validation of a Universal Values in Sport Instrument

Abstract

Universal values are one of the greatest aspects that can still be obtained through various sporting activities. Therefore, it is necessary to re-emphasize how this aspect can be assessed. This research aims to develop and test the validity of an instrument measuring universal values in sport, which includes five dimensions. With this instrument, it can later be used by physical education teachers in schools to find out the impact of learning carried out in relation to universal values. This is a descriptive review of inter-rater agreement and inter-rater reliability indices. It outlines the practical application and interpretation of these indices in research. The inter-rater agreement index assesses the extent of the responses of 7 expert raters. The data on agreement between raters and the reliability index were then processed to determine the validity and reliability results of the universal values in sports instrument being developed. The results of the research explain that the average validity value of each dimension was tested and the following results were obtained: Ethics and Morals (0.784), Knowledge and Understanding (0.805), Social (0.825), Psychological (0.855), and Leadership and Organization (0.835). The total reliability value of the five dimensions is 0.990, which shows a very high level of consistency and stability of this instrument. These results indicate that the instrument developed has strong reliability and validity in measuring various aspects of universal values in sports, so that it can be used effectively to support character building and skills development through sports activities. Keywords: Development, validation, interrater, reliability agreement, instruments, universal values.