Analysis of Impact of Marginal Expert Assessments on Integrated Expert Assessment

Burkov, Evgeniy; Paderno, P. I.; Siryk, Olga; Lavrov, Evgeniy; Pasko, Nadiia

doi:10.1109/scm50615.2020.9198772

Cited by 4 publications

(2 citation statements)

References 2 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…В исследованиях, посвященных методам экспертных оценок [1,2,11,12] и формированию экспертных групп [9,13,14], а также разработке поддерживающих экспертизу технических систем [15][16][17][18]…”

Section: подготовка экспертизы: сложившийся подходunclassified

“…Апробация и обсуждение отдельных принципов и идей методологии оценивания, в том числе положенных в основу рассмотренного подхода, были проведены на XXII и XXIII Международных конференциях по мягким вычислениям и измерениям (SCMʼ2019 и SCMʼ2020) [11,12,19] и вызвали интерес у специалистов в области подготовки и проведения различных экспертиз. Ранее ряд концептов, послуживших основой и получивших развитие в рассмотренном в данной работе подходе, был предложен и селективно использован при разработке системы профессионального отбора [20,21], в частности ее структуры и некоторых подсистем, а также программной реализации данной системы [22].…”

Section: результаты и перспективыunclassified

See 1 more Smart Citation

Methodological support of the working group in predicting the results of the classification expertise

Burkov¹,

Paderno²,

Sattorov³

et al. 2021

Naučno-teh. vestn. inf. tehnol. meh. opt.

View full text Add to dashboard Cite

The paper considers the specifics of the working group activity in preparation of expertise classification and the features of approaches for the choice of expert assessments and expert selection methods. The analysis focuses on potentially weak (illegibly) formalized customer requirements, which are quite typical for expertise classification. The used methods suffer from many drawbacks, which make them practically inapplicable in terms of planning, preparing and predicting the possible expertise classification reliability. The authors developed a new approach for predicting the reliability of the expertise classification at the stage of its preparation that depends on the reliability of the proposed expert group. The approach involves a probabilistic representation of the possible results of the work (namely, classification) of particular experts. The authors propose a number of probabilistic models (probabilistic matrices), which reflect the reliability (correctness) of the classification of certain objects both at the level of particular experts and at the level of entire expertise results. The set of procedures developed for a random group of experts allows obtaining probabilistic characteristics of the objects classification correctness when the group works as a part of an expert commission. The proposed approach can be used as a tool for working groups, which not only simplifies the process of an expert group selection, but also allows predicting the reliability of possible results and thereby makes it possible to take measures in advance in order to meet customer requirements. This approach can serve as a methodological basis for automating the problem solution in the expert selection process for the expertise classification at the level of its preparation, depending on customer requirements (restrictions). The proposed models and procedures will improve the efficiency of expertise classification, as well as save time for its preparation.

show abstract

Section: подготовка экспертизы: сложившийся подходunclassified

Section: результаты и перспективыunclassified

Methodological support of the working group in predicting the results of the classification expertise

Burkov¹,

Paderno²,

Sattorov³

et al. 2021

Naučno-teh. vestn. inf. tehnol. meh. opt.

View full text Add to dashboard Cite

show abstract

Issues of organization of expertise and problems of expert assessments

Paderno

Burkov

Lavrov

2020

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

In the course of preparation and carrying out of various expertises, in a number of cases methodological and organizational problems arise due to both the insufficiently high level of training of the working group members who develop the program and methodology of the expertise, and various restrictions, that could have a material impact on the credibility of the results. These restrictions generally are either financial in nature or are associated with a lack of sufficiently qualified and objective experts in the subject area. Objective: to develop a set of techniques that allow to expand the possibilities of using expert assessment methods in the preparation and carrying out of modern expertise in the presence of restrictions associated with external conditions. Methods: methods and approaches of system analysis, integration of individual expert assessments are used. Results: a comprehensive analysis of the problems faced by specialists in the organization of various expertises has been carried out. A number of techniques have been proposed that make it possible to significantly simplify the process of preparing and carrying out expertises but at the same time do not negatively affect the adequacy of the results obtained. Practical significance: the results obtained can be widely used in the course of various expertises, both in the process of selecting experts and in the process of direct obtaining and combining expert assessments. In addition, the use of obtained results in terms of different organizational and material restrictions will allow to expand the scope of possible applications of expert assessments. Prospects: the proposed approach and the set of techniques lay the methodological basis for creating a decision support system that will give specific recommendations to specialists involved in the development of program and methodology of the expertise at all stages of their activities. The introduction of such a system will greatly contribute to the simplification of various expertises, and will also increase confidence in their results.

show abstract