Pentacene organic thin-film transistors (OTFTs) using high-k Nd x Nb (1Àx) O gate dielectric with different Nb contents (x ¼ 1, 0.950, 0.908, and 0.877) are fabricated. The best OTFT has x ¼ 0.950, achieving a high carrier mobility of 1.95 cm 2 V À1 s À1 , small threshold voltage of À1.57 V, small sub-threshold swing of 0.13 V dec À1 , and small hysteresis of 0.13 V. Atomic force microscopy and X-ray photoelectron spectroscopy measurements reveal that the Nb doping can suppress the hygroscopicity of Nd oxide to produce a smoother dielectric surface, on which larger pentacene grains are grown to result in higher carrier mobility. The hysteresis of the OTFTs is attributed to donor-like traps associated with the hydroxide formed in Nd 2 O 3 after absorbing moisture and also acceptor-like traps (in the form of oxygen vacancies) induced by Nb incorporation.