This paper is devoted to experimental and theoretical studies of nonlinear propagation of a long-range surface plasmon polariton (LRSPP) in gold strip waveguides. The plasmonic waveguides are fabricated in house, and contain a gold layer, tantalum pentoxide adhesion layers, and silicon dioxide cladding. The optical characterization was performed using a high power picosecond laser at 1064 nm. The experiments reveal two nonlinear optical effects: nonlinear power transmission and spectral broadening of the LRSPP mode in the waveguides. Both nonlinear optical effects depend on the gold layer thickness. The theoretical model of these effects is based on the third-order susceptibility of the constituent materials. The linear and nonlinear parameters of the LRSPP mode are obtained, and the nonlinear Schrödinger equation is solved. The dispersion length is much larger than the waveguides length, and the chromatic dispersion does not affect the propagation of the plasmonic mode. We find that the third-order susceptibility of the gold layer has a dominant contribution to the effective third-order susceptibility of the LRSPP mode. The real part of the effective third-order susceptibility leads to the observed spectral broadening through the self-phase modulation effect, and its imaginary part determines the nonlinear absorption parameter and leads to the observed nonlinear power transmission. The experimental values of the third-order susceptibility of the gold layers are obtained. They indicate an effective enhancement of the thirdorder susceptibility for the gold layers, comparing to the bulk gold values. This enhancement is explained in terms of the change of the electrons motion.