Different material thickness with medium and high dielectric constant can impact the performance and reliability of high electron mobility transistor device. With varying the thickness of the passivation layer, the effect of it towards the device performance is still unclear. Two different insulator layers with a medium dielectric and a high dielectric constant namely Aluminium Nitride and Hafnium Oxide are used as passivation layer in AlGaN/GaN HEMT. Both material performance was simulated via COMSOL software by varying the thickness and the drain current output were compared. The passivation layer thickness of 10nm at Vds=6 V and Vgs=5 V, HfO2 outperforms AlN with the output drain current of 39 mA compared to 35 mA respectively. It was observed that HfO2 can attain higher threshold voltage, Vth as compared to the AlN because of the influence of its material properties that shows a direct proportional relationship between Vth and dielectric constant. Using high dielectric constant material like HfO2, we observe the ON-voltage gradually decreases as the thickness of the passivation layer increased. Out of all the thickness simulated for HfO2 and AlN, 10nm produced the highest drain current output instead of layer thickness of 20nm.