Computer-based neuropsychological assessments have many advantages over traditional neuropsychological assessments. However, limited data are available on the validity and reliability of computer-based assessments. The purpose of this study was to examine the reliability and validity of computer-based dementia assessment contents (Co-Wis). This study recruited 113 participants from Yeungnam University Medical Center in Daegu from June 2019 to December 2019 and received ethical approval. Participants were evaluated using standard and objective dementia cognitive test tools such as the Korean version of the Mini-Mental State Examination (K-MMSE), the Clinical Dementia Rating Scale (CDR), and the Standardized Seoul Neuropsychological Screening Battery-II (SNSB-II). To verify the effectiveness of Co-Wis, the concurrent validity, test–retest reliability (Pearson’s correlation coefficients), construct validity (Factor analysis), and signal detection analysis (ROC curve) were used. In most of the Co-Wis subtests, the concurrent validity and test–retest reliability showed statistically significant correlations (p < 0.05, p < 0.01). The factor analysis showed that Co-Wis assessed the most major cognitive areas (Tucker–Lewis Index (TLI) = 0.876, Comparative Fit Index (CFI) = 0.897, RMSEA = 0.88). Thus, Co-Wis appears clinically applicable and with high reliability and validity. In the future, we should develop tests to evaluate both standard data and big data-based machine learning.