Context. In a strongly stratified turbulent layer, a uniform horizontal magnetic field can become unstable and spontaneously form local flux concentrations due to a negative contribution of turbulence to the large-scale (mean-field) magnetic pressure. This mechanism, which is called negative effective magnetic pressure instability (NEMPI), is of interest in connection with dynamo scenarios in which most of the magnetic field resides in the bulk of the convection zone and not at the bottom, as is often assumed. Recent work using mean-field hydromagnetic equations has shown that NEMPI becomes suppressed at rather low rotation rates with Coriolis numbers as low as 0.1. Aims. Here we extend these earlier investigations by studying the effects of rotation both on the development of NEMPI and on the effective magnetic pressure. We also quantify the kinetic helicity resulting from direct numerical simulations (DNS) with Coriolis numbers and strengths of stratification comparable to values near the solar surface and compare it with earlier work at smaller scale separation ratios. Further, we estimate the expected observable signals of magnetic helicity at the solar surface. Methods. To calculate the rotational effect on the effective magnetic pressure we consider both DNS and analytical studies using the τ approach. To study the effects of rotation on the development of NEMPI we use both DNS and mean-field calculations of the three-dimensional hydromagnetic equations in a Cartesian domain. Results. We find that the growth rates of NEMPI from earlier mean-field calculations are well reproduced with DNS, provided the Coriolis number is below 0.06. In that case, kinetic and magnetic helicities are found to be weak and the rotational effect on the effective magnetic pressure is negligible as long as the production of flux concentrations is not inhibited by rotation. For faster rotation, dynamo action becomes possible. However, there is an intermediate range of rotation rates where dynamo action on its own is not yet possible, but the rotational suppression of NEMPI is being alleviated. Conclusions. Production of magnetic flux concentrations through the suppression of turbulent pressure appears to be possible only in the uppermost layers of the Sun, where the convective turnover time is less than two hours.