The automatic recognition of cracks is an essential requirement for the cost‐efficient maintenance of concrete structures, such as bridges, buildings, and roads. It should allow the localization and the determination of the crack type and the evaluation of the crack severity by providing information on the shape, orientation, and crack area and width. The first step in this direction is the automatized segmentation of cracks. This paper provides a concrete crack data set (370 images) and proposes two solutions that achieve the best results on two different crack data sets. Our first solution concerns the segmentation architecture. We provide an encoder–decoder‐based network with a particular interconnection of layers between the encoder and decoder parts that outperforms several other methods. In addition, this network is enhanced by squeeze‐and‐excitation blocks equipped with a modified sigmoid activation function. We introduce a stretch coefficient into the sigmoid function and declare it a trainable parameter, allowing more differentiated calibration of the feature map during network training. Our second solution concerns kernel initialization by transfer learning (TL). We propose the Copy‐Edit‐Paste Transfer Learning (CEP TL). By copying, geometric editing, and pasting crack masks onto new concrete background images, we generate thousands of semisynthetic images used to pretrain the network. This CEP TL method increases model performance with significant differences. For data set A (ours), we achieve F1‐scores 76.06 ± 0.06% without CEP TL and 92.32 ± 0.82% with CEP TL. For data set B (DeepCrack data set), we achieve F1‐scores 88.56 ± 0.01% without CEP TL and 90.59 ± 0.80% with CEP TL.