Understanding the spin dephasing mechanism is of fundamental importance in all potential applications of the spin qubit. Here we demonstrate a spin dephasing mechanism in a semiconductor quantum dot due to the 1/f charge noise. The spin-charge interaction is mediated by the interplay between the spin-orbit coupling and the asymmetrical quantum dot confining potential. The dephasing rate is proportional to both the strength of the spin-orbit coupling and the degree of the asymmetry of the confining potential. For parameters typical of the InSb, InAs, and GaAs quantum dots with a moderate well-height [Formula: see text] meV, we find the spin dephasing times are [Formula: see text] μs, 275 μs, and 55 ms, respectively. In particular, the spin dephasing can be enhanced by lowering the well-height. When the well-height is as small as [Formula: see text] meV, the spin depahsing times in the InSb, InAs, and GaAs quantum dots are decreased to [Formula: see text] μs, 18 μs, and 9 ms, respectively.