Linear and nonlinear resonant instabilities of charged-particle beams traveling in periodic quadrupole focusing channels are studied experimentally with a compact non-neutral plasma trap. The present experiments are based on the idea that the collective motion of a beam in an accelerator is physically similar to that of a one-component plasma in a trap. A linear Paul trap system named ''S-POD'' (simulator for particle orbit dynamics) was developed to explore a variety of space-charge-induced phenomena. To emulate lattice-dependent effects, periodic perturbations are applied to quadrupole electrodes, which gives rise to additional resonance stop bands that shift depending on the plasma density. It is confirmed that an mth-order resonance takes place when the corresponding tune of an mth-order collective mode m is close to a half integer.