Both polymer gels and crosslinked liquid-crystalline polymers (CLCPs) have unique properties to be developed as stimuli-responsive soft materials. While the former are suitable to be used in a wet circumstance, the latter are able to be applied as dry actuation materials. Among various stimulating sources, light has attracted more interest because it is a kind of clean energy which can be manipulated conveniently and controlled in situ. Photodeformable polymer gels and CLCPs undergo macroscopic shape changes such as contraction and bending induced by light, thus providing a photomechanical effect. This review will describe the advances in the field of photomanipulation regarding both the polymer gels and the CLCPs including the design of these smart materials, controlled actuation in response to light, and ability to tailor the materials properties for different applications.