The Kelvin-Helmholtz instability is analyzed at scales of micro-and nano-structures. We have studied the stability of a plane motionless layer of a two-layer incompressible viscous fluid using the Navier-Stokes equations for linear and nonlinear analyses. The effects of viscosity were assumed to occur at an interface with a flow inside the layers to be irrotational. A dispersion equation was developed for small perturbations, which is similar in form to the dispersion equation appeared in earlier papers, provided the short-wave approximation is applied. We first determined: 1) The dependence of the perturbation decrement for a viscous two-layer fluid that has two maxima, with the first maximum being within the wavelengths ranging from 100 to 300 nm and the second from 1 to 3 μm; 2) The approximate analytic dependence of wavenumber that contains the maximum for the perturbation decrement on input parameters for a problem (densities of both layers, their thicknesses, viscosity, surface tension, velocity of layer motion, a coefficient of resistance). It allows us to size up the emerging vortex structures under different conditions. The range of input parameters for the problem was determined, where two maxima relating to the dependence of disturbance decrement were observed. To verify the results after performing linear analysis, the Level Set Method was used for analyzing nonlinear equations. The results of calculations prove that linear analysis adequately describes how vortex structures in various sizes are formed.