The last decade has witnessed remarkable progress in the development of quantum technologies. Although fault-tolerant devices likely remain years away, the noisy intermediate-scale quantum devices of today may be leveraged for other purposes. Leading candidates are variational quantum algorithms (VQAs), which have been developed for applications including chemistry, optimization, and machine learning, but whose implementations on quantum devices have yet to demonstrate improvements over classical capabilities. In this Perspective, we propose a variety of ways that progress toward this potential crossover point could be informed by quantum optimal control theory. To set the stage, we identify VQAs and quantum optimal control as formulations of variational optimization at the circuit level and pulse level, respectively, where these represent just two levels in a broader hierarchy of abstractions that we consider. In this unified picture, we suggest several ways that the different levels of abstraction may be connected, in order to facilitate the application of quantum optimal control theory to VQA challenges associated with ansatz selection, optimization landscapes, noise, and robustness. A major theme throughout is the need for sufficient control resources in VQA implementations; we discuss different ways this need can manifest, outline a variety of open questions, and conclude with a look to the future.