The main goal of this paper is the proposal of a key-based steganographic system in which the ratio of capacity and image quality metrics that represents the stego object while reducing the detectability of hidden content was improved. The main contribution of the proposed steganographic system is a new algorithm for selecting stego areas. The area selection algorithm is based on clustering the pixels of the cover object into a predetermined number of clusters. The goal of this selection of areas (clusters) is to group as many homogeneous parts of the image as possible in order to cover these areas with as few rectangular shapes as possible. Since the data on the defined rectangles represent the key of the system, the capacity of the additional secret channel is minimized in this way. On the obtained stego-carriers, an embedding of test random content is performed in order to estimate its detectability. By combining the proposed area selection method with the Minimal Decimal Difference steganographic method, a system was created with an optimal trade-off between detectability of secret content, quality and capacity of the carrier, and the length of the stego-key. Finally, a comparison of the obtained results with relevant adaptive steganographic methods is presented. The proposed concept obtains its verification in one practical system for secure file transfer of controlled cryptographic strength.