Context. Relaxing the assumption of Local Thermodynamic Equilibrium (LTE) in modelling stellar spectra is a necessary step to determine chemical abundances better than about 10 % in late-type stars. Aims. In this paper we describe our efforts gearing up to perform multi-element (Na, Mg, K, and Ca) Non-LTE (NLTE) calculations that can be applied to the APOGEE survey. Methods. The new version of TLUSTY allows for the calculation of restricted NLTE in cool stars using pre-calculated opacity tables. We demonstrate that TLUSTY gives consistent results with MULTI, a well-tested code for NLTE in cool stars. We use TLUSTY to perform LTE and a series of NLTE calculations using all combinations of 1, 2, 3 and the 4 elements mentioned above simultaneously in NLTE. Results. In this work we take into account how departures from LTE in one element can affect others through changes in the opacities of Na, Mg, K, and Ca. We find that atomic Mg, which provides strong UV opacity, and exhibits significant departures from LTE in the low-energy states, can impact the NLTE populations of Ca, leading to abundance corrections as large as 0.07 dex. The differences in the derived abundances between the single-element and the multi-element cases can exceed those between the single-element NLTE determinations and an LTE analysis, warning that this is not always a second-order effect. By means of detailed tests for three stars with reliable atmospheric parameters (Arcturus, Procyon and the Sun) we conclude that our NLTE calculations provide abundance corrections that can amount in the optical up to 0.1, 0.2 and 0.7 dex for Ca, Na and K, but LTE is a good approximation for Mg. In the H-band, NLTE corrections are much smaller, and always under to 0.1 dex. The derived NLTE abundances in the optical and in the IR are consistent. For all four elements, in all three stars, NLTE line profiles fit better the observations than the LTE counterparts. Conclusions. The atomic elements in ionisation stages where over-ionisation is an important NLTE mechanism are likely affected by departures from LTE in Mg . Special care must be taken with the collisions adopted for high-lying levels when calculating NLTE profiles of lines in the H-band. The derived NLTE corrections in the optical and in the H-band differ, but the derived NLTE abundances are consistent between the two spectral regions.