Purpose: MRI is commonly used in follow up of high grade glioma. Our purpose is to assess the interrater agreement on the increasingly used visual qualitative assessment of various conventional and advanced MR techniques in the setting of treated high grade glioma in comparison to the well established quantitative measurements. Methods: We prospectively enrolled HGG patients who underwent reresection of a new enhancing lesion on post-treatment 3T MR examination including DWI, DCE and DSC sequences. Two neuroradiologists objectively assessed the diffusion and perfusion maps by placing ROI on representative post-processed maps. They subjectively assessed the post-contrast, perfusion and diffusion sequences. Interrater agreement and concordance correlation coefficient were calculated. Results: Twenty-eight lesions were included. The interrater agreement on the qualitative assessment was good for k-trans (k = 0.73), moderate for Vp (k = 0.52), fair for AUC and Ve maps (k = 0.37 and 0.21), fair for corrected CBV (k = 0.39) and poor for the enhancement pattern and presence of diffusion restriction (k = 0.02 and 0.07). The concordance between the quantitative measurements was substantial for AUC and Vp (ρc = 0.98 and 0.97), moderate for k-trans and corrected CBV (ρc = 0.94) and poor for Ve and ADC (ρc = 0.86 and 0.24). Conclusion: While the quantitative measurements of DSC and DCE perfusion maps show satisfactory inter-rater agreement, the qualitative assessment has lower interobserver agreement and should not be relied upon solely in the interpretation. Similarly, the suboptimal inter-rater agreement on the interpretation of enhancement pattern and diffusion restriction potentially limits their usefulness in differentiating glioma recurrence from treatment related changes.