In this paper we present a regularized Lagrangian finite point method (RLFPM) for the numerical simulation of incompressible viscous flows. A Lagrangian finite point scheme is applied to the projection method for the incompressible Navier-Stokes equations. The approximation of spatial derivatives is obtained by the weighted least squares method. The pressure Poisson equation with Neumann boundary condition is solved by a stabilized finite point method. A key aspect of the present approach is the periodic redistribution of the particle locations, which are being distorted by the flow. Again, weighted least squares approximation is implemented to interpolate the properties of the old particles onto the new particle locations. With the proposed regularization technique, problems associated with the flow-induced irregularity of particle distribution in the Lagrangian finite point scheme are circumvented. Three numerical examples, Taylor-Green flow, lid-driven flow in a cavity and flow through a periodic lattice of cylinders, are presented to validate the proposed methodology. The problem of extra diffusion caused by regularization is discussed. The results demonstrate that RLFPM is able to perform accurate and stable simulations of incompressible viscous flows.