An element-block, which is the heteroatom-containing functional minimum unit, is a versatile building block for constructing functional polymers. In this focus review, we explain the new concept for developing stimuli-responsive luminochromic materials based on "the frustrated" element-block. When the element-blocks which show relatively larger degree of structural relaxation in the excited state are structurally restricted, we regarded that these molecules are in "the frustrated state". Recently, we discovered unique stimuli-responsive luminochromic behaviors from "the frustrated element-blocks". As a representative example, stimuli-responsive properties of "the frustrated boron clusters" are initially illustrated. The series of luminochromic properties, such as mechano-and thermochromism are described. Next, we designed the fused complexes for suppressing intramolecular motions to obtain "frustrated boron complexes". Not only constant emission in solution and solid but also mechanochromic luminescent materials have been obtained. Finally, "the frustrated state" by the polymerization is explained. Unique luminochromic behaviors, such as thermochromism in solution, mechanochromism in solid and vapochromism were observed from the polymers. Mainly from our recent works, stimuli-responsive luminochromic behaviors of "the frustrated element-block"containing polymers and their mechanisms are illustrated.