The transient coupled radiative and conductive heat transfer in a semitransparent composite under the complex boundary conditions is investigated by the ray tracing method in combination with Hottel's zonal method and the control-volume method. The composite is composed of two plane layers of nonscattering semitransparent media with the different thermophysical properties in each layer. Both boundary surfaces and the internal interface are semitransparent. The reflections are assumed diffuse or specular. The transient temperature distributions in the composite are obtained for the combined thermal boundary conditions of incident radiation and convective heat transfer. Under diffuse reflection, the results in this paper are separately compared with the steady and transient results of previous work. The comparison shows the reliability and the high calculating accuracy of the formulas derived in this paper. The present analysis includes the effects of the optical thickness, the conduction-radiation parameter, the spectral property and the reflective mode on the transient temperature distributions.