A new cell-vortex unstructured finite volume method for structural dynamics is assessed for simulations of structural dynamics in response to fluid motions. A robust implicit dual-time stepping method is employed to obtain time accurate solutions. The resulting system of algebraic equations is matrix-free and allows solid elements to include structure thickness, inertia, and structural stresses for accurate predictions of structural responses and stress distributions. The method is coupled with a fluid dynamics solver for fluid-structure interaction, providing a viable alternative to the finite element method for structural dynamics calculations. A mesh sensitivity test indicates that the finite volume method is at least of second-order accuracy. The method is validated by the problem of vortex-induced vibration of an elastic plate with different initial conditions and material properties. The results are in good agreement with existing numerical data and analytical solutions. The method is then applied to simulate a channel flow with an elastic wall. The effects of wall inertia and structural stresses on the fluid flow are investigated.