Density functional theory calculations were carried out to study the mechanisms of Pd-catalyzed reactions of alkynes with tertbutyl isocyanide forming pyrroles. In the case of di-palladium catalysis, the calculations indicated that alkyne insertion, isocyanide insertion, intramolecular cyclization, isocyanide insertion, isomerization and ligand substitution were involved in the mechanism. The intramolecular cyclization process was calculated to be the rate-determining step in the catalytic cycle. The substituent effects on the regioselectivity of reaction were also analyzed. In addition, detailed calculations on monopalladium catalysis showed that the activity of mononuclear catalysis was not as high as that of di-palladium catalysis. These calculation results were in good agreement with the experimental results.