The emergent photoactive materials through photochemistry make it possible to directly convert photon energy to mechanical work. There is much recent work in developing appropriate materials and a promising new system is semi-crystalline polymers of the photoactive molecule azobenzene. We develop a phase field model with two order parameters for the crystal-melt transition and the trans-cis photo-isomerization to understand such materials, and the model describes the rich phenomenology. We find that the photoreaction rate depends sensitively on temperature: at temperatures below the crystal-melt transition temperature, photoreaction is collective, requires a critical light intensity and shows an abrupt first order phase transition manifesting nucleation and growth; at temperatures above the transition temperature, photoreaction is independent and follows first order kinetics. Further, the phase transition depends significantly on the exact forms of spontaneous strain during the crystal-melt and trans-cis transitions. A non-monotonic change of photo-stationary cis ratio with increasing temperature is observed accompanied by a reentrant crystallization of trans below the melting temperature. A pseudo phase diagram is subsequently presented with varying temperature and light intensity along with the resulting actuation strain.These insights can assist the further development of these materials.