The internal structures and the hygroscopicity of bio-based boards consisting of giant reed (Arundo donax L.) and hazelnut shells as bio-aggregates, and a sodium silicate solution as the adhesive, were investigated. The aim was to evaluate the influence of each material (the bio-aggregates and adhesive) and their distributions in the boards on the final performance. By carrying out X-ray computed tomography, the internal structures and the porosities of the boards were determined, allowing important considerations of their hygroscopicity. The voids’ percentages were between 26% and 36% of the total volume of the composites. Both the materials and the composites demonstrated high hygroscopicity. In particular, the mixtures of the bio-aggregates and the sodium silicate allowed reaching a moisture buffering value of 7.44 g/(m2%RH) for the A. donax-based composite, 3.86 g/(m2%RH) for the hazelnut-shell-based composite, and 4.65 g/(m2%RH) for the mixture-based composite. Besides the identification of the contributions of the materials, a detailed discussion of the assessed properties was carried out to use these bio-based boards in vernacular historic construction. The results show how the aggregate type and the adhesive content affected the final behavior, demonstrating the importance of a conscious material choice. Furthermore, helpful information for the future development of these types of bio-based boards and their possible optimization was provided.