The immersed boundary lattice Boltzmann method (IB-LBM) has been widely used in the simulation of fluid-solid interaction and particulate flow problems, since proposed in 2004. However, it is usually a non-trivial task to retain the flexibility and the accuracy simultaneously in the implementation of this approach. Based on the intrinsic nature of the LBM, we propose a simple and second-order accurate IB-LBM in this paper, where the no-slip boundary condition on the fluid-solid interface is directly imposed by iteratively correcting the distribution function near the boundary, markedly similar to the treatment of boundary condition in the original LBM. Therefore, there are no additional efforts needed to construct an interfacial force model (such as the spring, feedback and direct force models) and absorb this force into the framework of LBM. It is worth mentioning that we retain the discrete delta function for the connection between the Lagrangian and Eulerian meshes, thus preserving the flexibility of the conventional IB-LBM. Second-order accuracy of the present IB-LBM is reasonably confirmed in the simulation of cylindrical Couette flow, while the conventional approach is generally a first-order scheme.The present method is first validated in the flow past a fixed cylinder. No streamline penetration is found, indicating the exactly enforcement of the no-slip boundary condition. Furthermore, in the simulations of one and two particles settling under gravity, good agreements can be observed comparing with the data available in the literature.