An Approach to Generating Reference Information for Technology Evaluation

Abstract

A system for technology evaluation is essential for successfully implementing a technology-based financial support system. Technology evaluation has generally relied on the qualitative evaluation performed by the relevant experts. When evaluating the technologies that a certain target firm possesses, the previous evaluation results for other firms that are similar to the target firm are used together for the purpose of improving the efficiency of the qualitative evaluation. To do this, technology evaluation institutes including, KOTEC, have presented a way to create a peer group and generate reference information in order to provide a clear guidance to the evaluators. However, the current approaches have limitations in that they cannot explore the detailed features of the individual firms. Therefore, this study proposes a systematic approach to generate reference information that facilitates efficient technology evaluation. We first create a peer group by collecting the relevant firms that have similarities with a certain target firm, and then measure the internal and external similarities between the target firm and all of the firms included in the peer group. We define the average value of similarities according to each evaluation rating as density, and finally generate the distribution and the descriptive statistics for the density as reference information. We expect that this study can contribute to improving the efficiency of qualitative evaluation work by provide practical reference information. Furthermore, the reliability of the technology evaluation will also be improved by reducing the difference in the evaluation results due to the individual differences of the evaluator.

A Hybrid Approach to Building Peer Groups for Technology Evaluation to Generate Reference Information

“…When using qualitative indicators, evaluators have used the previous evaluation results of the peer group as a guideline for the current evaluation [8]. The current approach simply groups firms belonging to the same industry or technology category to build peer groups, but it cannot explore the detailed features of individual firms [5]. Another approach builds peer groups by identifying firms that have similar internal and external structures to a focal firm, but it also measures the firm similarity only from the perspective of the firm structure.…”

“…Second, the structural similarity between the relevant firms and the focal firm are computed and the two similarities are combined to produce the aggregate similarity. The previous approach [5] has also recognized it and adopted a filtering method that leaves only firms classified in the same industry category. It can be quite reasonable because firms belonging to the same industry category generally tend to share many common and similar characteristics [9].…”

“…The quantitative one is usually used as a means to facilitate the qualitative one. It means that the technology evaluation mainly relies on the qualitative judgement of experts, which causes the problem of inconsistent evaluation results [5]. Even for the same technology, considerably different evaluation results can be produced depending on who evaluates it, which lowers the reliability of the technology evaluation.…”

“…When conducting a new evaluation of the technology possessed by a particular focal firm, the previous evaluation results for other firms similar to the focal firm are utilized together to improve the efficiency and reliability of qualitative evaluation [7]. The previous results can be seen as reference information for new evaluation of the focal firm [5]. However, the existing method that has been currently used has a limitation in that it cannot consider the specific features of individual firms.…”

“…Even firms classified in the same category have quite different characteristics from each other. To remedy this problem, an approach has been presented to build peer groups by measuring the internal and external similarities between individual firms [5]. It has proven to be able to generate more useful reference information than the classification similarity-based approach does.…”