The detection of polymorphic and metamorphic malware is a critical cybersecurity challenge due to its ability to
evade detection by existing cyber defense systems by automatically modifying its own code and/or structure. In this
context, an approach to the detection of polymorphic and metamorphic malware is proposed, which is based on the
determination of an invariant component for each known type of malware during the analysis of its behavior. The essence
of this approach is to define such an area of behavior that remains unchanged for a specific type of malicious software,
regardless of the modifications made. To find the specified invariant component in the behavior of malware for each of
its types, a set of values of the original feature space is described by fuzzy linguistic terms in order to obtain a set of fuzzy
production rules for each type of malware. The next step is to determine the fuzzy invariant component for each known
type of malicious software in the form of a fuzzy subset of features from the set of fuzzy production rules obtained in the
previous step by means of genetic algorithms. The proposed model makes it possible to significantly increase the accuracy
of detection of polymorphic and metamorphic software based on behavioral characteristics characteristic of already
classified samples, which, in turn, contributes to increasing the overall effectiveness of the cyber security system.