The out-of-time-order correlator (OTOC) is considered as a measure of quantum chaos. We formulate how to calculate the OTOC for quantum mechanics with a general Hamiltonian. We demonstrate explicit calculations of OTOCs for a harmonic oscillator, a particle in a one-dimensional box, a circle billiard and stadium billiards. For the first two cases, OTOCs are periodic in time because of their commensurable energy spectra. For the circle and stadium billiards, they are not recursive but saturate to constant values which are linear in temperature. Although the stadium billiard is a typical example of the classical chaos, an expected exponential growth of the OTOC is not found. We also discuss the classical limit of the OTOC. Analysis of a time evolution of a wavepacket in a box shows that the OTOC can deviate from its classical value at a time much earlier than the Ehrenfest time, which could be the reason of the difficulty for the numerical analyses to exhibit the exponential growth.