Probabilistic power flow (PPF) analysis is critical to power system operation and planning. PPF aims at obtaining probabilistic descriptions of the state of the system with stochastic power injections (e.g., renewable power generation and load demands). Given power injection samples, numerical methods repeatedly run classic power flow (PF) solvers to find the voltage phasors. However, the computational burden is heavy due to many PF simulations. Recently, many data-driven based PF solvers have been proposed due to the availability of sufficient measurements. This paper proposes a novel neural network (NN) framework which can accurately approximate the non-linear AC-PF equations. The trained NN works as a rapid PF solver, significantly reducing the heavy computational burden in classic PPF analysis. Inspired by residual learning, we develop a fully connected linear layer between the input and output in the multilayer perceptron (MLP). To improve the NN training convergence, we propose three schemes to initialize the NN weights of the shortcut connection layer based on the physical characteristics of AC-PF equations. Specifically, two model-based methods require the knowledge of system topology and line parameters, while the purely data-driven method can work without power grid parameters. Numerical tests on five benchmark systems show that our proposed approaches achieve higher accuracy in estimating voltage phasors than existing methods. In addition, three meticulously designed initialization schemes help the NN training process converge faster, which is appealing under limited training time.INDEX TERMS Data-driven, neural network, probabilistic power flow, physics-guided initialization, residual learning.