Efficient verification of the functioning of quantum devices is a key to the development of quantum technologies, but is a daunting task as the system size increases. Here we propose a simple and general framework for verifying unitary transformations that can be applied to both individual quantum gates and gate sets, including quantum circuits. This framework enables efficient verification of many important unitary transformations, including but not limited to all bipartite unitaries, Clifford unitaries, generalized controlled-Z gates, generalized CNOT gates, and CSWAP gate. The sampling complexity increases at most linearly with the system size and is often independent of the system size. Moreover, little overhead is incurred even if one can only prepare Pauli eigenstates and perform local measurements. Our approach is applicable in many scenarios in which randomized benchmarking (RB) does not apply and is thus instrumental to quantum computation and many other applications in quantum information processing.