The use of waste plants in the production of building materials is consistent with the principles of sustainable development. One of the ideas involves using hemp shives as an aggregate for the production of a composite used as a filling of the timber frame construction of the walls. The most important disadvantage of using the building materials based on organic components is their susceptibility to the water influence. The wall material is exposed to rising groundwater. The research part of the paper presented the preparation method and the investigation of the hemp-perlite-lime composites. Flexural and compressive strength, apparent density, total porosity, thermal conductivity, and mass absorptivity were examined. The main research part pertained to the analysis of capillary uptake occurrence in the composites, being the important phenomenon present in the external walls. The study on this phenomenon was carried out using the technique of indirect moisture evaluation—Time Domain Reflectometry (TDR). The indirect readouts were additionally verified with the traditional evaluation using the gravimetric method based on the PN-EN 1925 standard. The study proved that the tested composites were characterized by low apparent density, thermal conductivity, strength parameters, high total porosity, and mass absorptivity. The partial replacement of hemp shives by expanded perlite had a beneficial effect on the tested properties of composites.