The liquid–solid concave‐wall jet existed in the vertical cylinder separator with tangential inlet; the instability near concave‐wall was the key factor for centrifugal separation. The effects of jet flowrate and wall curvature radius on the instability of solid‐particles were explored by experiment and numerical simulation. The results showed that the separation efficiency of particles decreased with the increase of centrifugal separation factor. The liquid–solid concave‐wall jet flow in separator was unstable in this study. The decay rate of jet maximum velocity on the concave‐wall was larger than that on plane‐wall jet. The ratio of half‐width value to wall curvature radius was about 0.08 at the circumferential direction 90°, which coincided with the critical value of centrifugal instability in strongly concave‐wall jet. The pressure, velocity, and vorticity fluctuated periodically along streamwise direction in separator. The fluctuation period was linearly related to the wall curvature, and the radial range of fluctuation was only half of jet width. The paired longitudinal vortices formed along the flow direction, and the longitudinal vortices merged at length 0.74 m from the inlet cross‐section along circumferential direction. After the position of vortices merged, the particle trajectory oscillated obviously.