BackgroundLung and diaphragm ultrasound methods have recently been introduced to predict the outcome of weaning from mechanical ventilation (MV). The aim of this study is to assess the reliability and accuracy of these techniques for predicting successful weaning in critically ill adults.MethodsWe conducted two studies: a cross-sectional interobserver agreement study between two sonographers and a prospective cohort study to assess the accuracy of lung and diaphragm ultrasound for predicting weaning and extubation outcome. For the interobserver agreement study, we included 50 general critical care patients who were consecutively admitted to the ICU. For the predictive accuracy study, we included consecutively 69 patients on MV who were ready for weaning. We assessed interobserver agreement of ultrasound measurements, using the weighted kappa coefficient for LUSm score (modified lung ultrasound score) and the intraclass correlation coefficient (ICC) and Bland–Altman method for TI (diaphragm thickening index). We assessed the predictive value of LUSm and TI in weaning outcome by plotting the corresponding ROC curves.ResultsWe found adequate interobserver agreement for both LUSm (weighted kappa 0.95) and TI (ICC 0.78, difference according to Bland–Altman analysis ± 12.5%). LUSm showed good-moderate discriminative power for successful weaning and extubation (area under the ROC curve (AUC) for successful weaning 0.80, and sensitivity and specificity at optimal cut-off point 0.76 and 0.73, respectively; AUC for successful extubation 0.78, and optimal sensitivity and specificity 0.76 and 0.47, respectively. TI was more sensitive but less specific for predicting successful weaning (AUC 0.71, optimal sensitivity and specificity 0.93 and 0.48) and successful extubation (AUC 0.76, optimal sensitivity and specificity 0.93 and 0.58). The area under the ROC curve for predicting weaning success was 0.83 for both ultrasound measurements together.ConclusionsInterobserver agreement was excellent for LUSm and moderate-good for TI. A low TI value or high LUSm value indicates high risk of weaning failure.Electronic supplementary materialThe online version of this article (10.1186/s13089-018-0094-3) contains supplementary material, which is available to authorized users.