In a condensed matter system, phonons and plasmons are well-known quasiparticles that represent unusual dispersion behavior of energy and momentum at nanoscales. In a nanoscale Mo/Si multilayer structure, phonon modes in Raman scattering indicated the coexistence of crystalline Si (c-Si) nanoclusters within an amorphous silicon (a-Si) matrix. The TO mode was red-shifted with a decrease in the nanocluster size of Si in nanolayer films. This was associated with the momentum of phonons and it is fundamentally correlated to phonon confinement. The correlation length of the Si network was significantly smaller in a-Si and the TO mode broadened asymmetrically and redshifted due to localized phonon density of state. Consequently, with a decrease in the thickness of the Si layer, blue shifts of plasmon energy for Mo 3d, Mo 4p, and Si 2p spectra were observed in X-ray photoelectron spectroscopy. Plasmon energy of the c-Si nanocluster was related to the forbidden gap, which increased with a decrease in cluster size. The concept of quantum confinement of phonon and electron states was used to determine the size of the c-Si nanoclusters in the a-Si matrix.