Planting seedlings remains the most popular method of forest restoration, but passive restoration may become a new promise to reduce costs, despite concerns about its efficacy. Here, we present a study of a riparian forest recovery program in the Southern Atlantic Forest, Brazil. The objective was to compare planting seedlings and passive restoration eight years since restoration started and assess the influence of cattle presence in the restoring areas. We analyzed the structure, diversity, and composition of the woody community in the upper (canopy) and lower strata (understory) versus six environmental variables, including edaphic, landscape and geographical variables, restoration method (planting seedlings versus passive restoration), presence of cattle inside the restoration areas, and presence of adjacent crop plantations. Passive restoration generated a more diverse woody community and a higher proportion of climax and zoochoric species in the lower stratum than active restoration. Cattle presence was significantly associated with lower species richness and density of plants, as well as a higher proportion of pioneer species and low proportion of zoochoric and climax species. The results suggest that cattle can delay forest succession, besides reducing diversity. Passive restoration should be considered suitable for the region, and the efficiency of cattle exclusion needs to be improved.