Hybrid energy methods such as QM/MM and ONIOM, that combine different levels of theory into one calculation, have been very successful in describing large systems. Geometry optimization methods can take advantage of the partitioning of these calculations into a region treated at a quantum mechanical (QM) level of theory and the larger, remaining region treated by an inexpensive method such as molecular mechanics (MM). A series of microiterations can be employed to fully optimize the MM region for each optimization step in the QM region. Cartesian coordinates are used for the MM region and are chosen so that the internal coordinates of the QM region remain constant during the microiterations. The coordinates of the MM region are augmented to permit rigid body translation and rotation of the QM region. This is essential if any atoms in the MM region are constrained, but it also improves the efficiency of unconstrained optimizations. Because of the microiterations, special care is needed for the optimization step in the QM region so that the system remains in the same local valley during the course of the optimization. The optimization methodology with microiterations, constraints, and step-size control are illustrated by calculations on bacteriorhodopsin and other systems.

Geometry optimization has become an essential part of quantum-chemical computations, largely because of the availability of analytic first derivatives. Quasi-Newton algorithms use the gradient to update the second derivative matrix (Hessian) and frequently employ corrections to the quadratic approximation such as rational function optimization (RFO) or the trust radius model (TRM). These corrections are typically carried out via diagonalization of the Hessian, which requires O(N3) operations for N variables. Thus, they can be substantial bottlenecks in the optimization of large molecules with semiempirical, mixed quantum mechanical/molecular mechanical (QM/MM) or linearly scaling electronic structure methods. Our O(N2) approach for solving the equations for coordinate transformations in optimizations has been extended to evaluate the RFO and TRM steps efficiently in redundant internal coordinates. The regular RFO model has also been modified so that it has the correct size dependence as the molecular systems become larger. Finally, an improved Hessian update for minimizations has been constructed by combining the Broyden–Fletcher–Goldfarb–Shanno (BFGS) and (symmetric rank one) SR1 updates. Together these modifications and new methods form an optimization algorithm for large molecules that scales as O(N2) and performs similar to or better than the traditional optimization strategies used in quantum chemistry.

The geometry optimization using direct inversion in the iterative subspace (GDIIS) has been implemented in a number of computer programs and is found to be quite efficient in the quadratic vicinity of a minimum. However, far from a minimum, the original method may fail in three typical ways: (a) convergence to a nearby critical point of higher order (e.g. transition structure), (b) oscillation around an inflection point on the potential energy surface, (c) numerical instability problems in determining the GDIIS coefficients. An improved algorithm is presented that overcomes these difficulties. The modifications include: (a) a series of tests to control the construction of an acceptable GDIIS step, (b) use of a full Hessian update rather than a fixed Hessian, (c) a more stable method for calculating the DIIS coefficients. For a set of small molecules used to test geometry optimization algorithms, the controlled GDIIS method overcomes all of the problems of the original GDIIS method, and performs as well as a quasi-Newton RFO (rational function optimization) method. For larger molecules and very tight convergence, the controlled GDIIS method shows some improvement over an RFO method. With a properly chosen Hessian update method, the present algorithm can also be used in the same form to optimize higher order critical points.