In this paper, we study the impact of stealthy attacks on the Cyber-Physical System (CPS) modeled as a stochastic linear system. An attack is characterised by a malicious injection into the system through input, output or both, and it is called stealthy (resp. strictly stealthy) if it produces bounded changes (resp. no changes) in the detection residue. Correspondingly, a CPS is called vulnerable (resp. strictly vulnerable) if it can be destabilized by a stealthy attack (resp. strictly stealthy attack). We provide necessary and sufficient conditions for the vulnerability and strictly vulnerability. For the invulnerable case, we also provide a performance bound for the difference between healthy and attacked system. Numerical examples are provided to illustrate the theoretical results.