The aim of this study was to analyze the reliability of texture index (TI) calculations using two different approaches. First, we calculated texture parameters on synthetically constructed images using four different biomedical software tools (CGITA, InterView Fusion, Matlab, MaZda). Second, we investigated the reliability of texture parameters, particularly how the texture indices diverge between two similar images with substantially different texture. We generated five different heterogeneous synthetic images, thereafter, histogram-based and co-occurrence matrix features were calculated. The co-occurrence based indices were computed after two (8 and 64) different gray scale normalizations. For the reliability test, we compared 22 texture indices using a histological slice of the brain and Michelangelo's painting, and the gray level dependence was also analyzed. The histogram-based parameters of all images and from all software were very similar. Differences were found in the co-occurrence based indices after both gray level image normalizations. The reliability tests showed that from 22 parameters only 5 texture indices changed more than 20%, and at least 64 normalization levels were necessary for acceptable results. Our results underline that in medical multicenter studies it is especially critical to use the same software package. Some parameters do not reliably reflect changes, so texture analysis (TA) should be used with caution.