Current trends in bio-medicine include research synthesis and dissemination of bioinformation by means of health (bio) information technology (H[b] IT). Research must secure the validity and reliability of assessment tools to quantify research quality in the pursuit of the best available evidence. Our concerted work in this domain led to the revision of three instruments for that purpose, including the stringent characterization of inter-rater reliability and coefficient of agreement. It is timely and critical to advance the methodological development of the science of research synthesis by strengthening the reliability of existing measure of research quality in order to ensure H[b] IT efficacy and effectiveness.