In this paper, we consider inverse time-harmonic acoustic and electromagnetic scattering from locally perturbed rough surfaces in three dimensions. The scattering interface is supposed to be the graph of a Lipschitz continuous function with compact support. It is proved that an acoustically sound-soft or sound-hard surface can be uniquely determined by the far-field pattern of infinite number of incident plane waves with distinct directions. Moreover, a single point source or plane wave can be used to uniquely determine a scattering surface of polyhedral type. These uniqueness results apply to Maxwell equations with the perfectly conducting boundary condition. Our arguments rely on the mixed reciprocity relation in a half space and the reflection principle for Helmholtz and Maxwell equations.