Extending our previous work on time optimal quantum state evolution [A. Carlini, A. Hosoya, T. Koike and Y. Okudaira, Phys. Rev. Lett. 96, 060503 (2006)], we formulate a variational principle for finding the time optimal realization of a target unitary operation, when the available Hamiltonians are subject to certain constraints dictated either by experimental or by theoretical conditions. Since the time optimal unitary evolutions do not depend on the input quantum state this is of more direct relevance to quantum computation. We explicitly illustrate our method by considering the case of a two-qubit system self-interacting via an anisotropic Heisenberg Hamiltonian and by deriving the time optimal unitary evolution for three examples of target quantum gates, namely the swap of qubits, the quantum Fourier transform and the entangler gate. We also briefly discuss the case in which certain unitary operations take negligible time.