In this article, a decentralized control strategy is presented for harmonic drive-based modular and reconfigurable robots with uncertain environment contact. Unlike conventional methods that rely on robot-environment contact model or force/torque sensing, this article addresses the problem of controlling modular and reconfigurable robots in contact with uncertain environment using only encoder data of each joint module. By employing a control-oriented harmonic drive model, the dynamic model of modular and reconfigurable robot is formulated as a synthesis of interconnected subsystems, in which the interconnected joint couplings are with small magnitudes. Based on the integral sliding mode control technique and the adaptive super-twisting algorithm, the decentralized controller is designed to compensate model uncertainty in which the up-bound is unknown. The stability of the modular and reconfigurable robot system is proved using Lyapunov theory. Finally, simulations are conducted for 2-degree-of-freedom modular and reconfigurable robots with different configurations under the situations of dynamic contact and collision to investigate the advantage of the proposed approach.