We derive a fundamental trade-off relation between accuracy of implementing a desired unitary evolution using a restricted set of free unitaries and the size of the assisting system, in terms of the resource generating/losing capacity of the target unitary. In particular, this relation implies that, for any theory that is equipped with a resource measure satisfying lenient conditions, any resource changing unitary cannot be perfectly implemented by a free unitary applied to system and environment if the environment has finite dimensions. Our results are applicable to a wide class of resources including energy, asymmetry, coherence, entanglement, and magic, imposing ultimate limitations inherent in those important physical settings, as well as providing new insights into operational restrictions in general resource theories.