It has been long known that numerous halide and oxide perovskites can have non-ideal octahedra, showing tilting, rotation, and metal atom displacements. It has also been known that compounds that have at low temperatures a single structural motif ("monomorphous structures") could become disordered at higher temperatures, resulting in non-ideal octahedra as an entropy effect. What is shown here is that in many cubic halide perovskites and some oxides compounds a distribution of different low-symmetry octahedra ("polymorphous networks") emerge already from the minimization of the systems internal energy, i.e., they represent the intrinsic, preferred low temperature pattern of chemical bonding. Thermal disorder effects build up at elevated temperatures on top of such low temperature polymorphous networks. Compared with the monomorphous counterparts, the polymorphous networks have lower predicted total energies (enhanced stability), larger band gaps and dielectric constants now dominated by the ionic part, and agrees much more closely with the observed pair distribution functions. The nominal cubic perovskites (Pm-3m) structure deduced from X-Ray diffraction is actually a macroscopically averaged, high symmetry configuration, which should not be used to model electronic properties, given that the latter reflect a low symmetry local configuration.