Developing reliable ultrasonic-guided wave monitoring systems requires a significant amount of inspection data for each application scenario. Experimental investigations are fundamental but require a long period and are costly, especially for real-life testing. This is exacerbated by a lack of experimental data that includes damage. In some guided wave applications, such as pipelines, it is possible to introduce artificial damage and perform lab experiments on the test structure. However, in rail track applications, laboratory experiments are either not possible or meaningful. The generation of synthetic data using modelling capabilities thus becomes increasingly important. This paper presents a variational autoencoder (VAE)-based deep learning approach for generating synthetic ultrasonic inspection data for welded railway tracks. The primary aim is to use a VAE model to generate synthetic data containing damage signatures at specified positions along the length of a rail track. The VAE is trained to encode an input damage-free baseline signal and decode to reconstruct an inspection signal with damage by adding a damage signature on either side of the transducer by specifying the distance to the damage signature as an additional variable in the latent space. The training data was produced from a physics-based model that computes virtual experimental response signals using the semi-analytical finite element and the traditional finite element procedures. The VAE reconstructed response signals containing damage signatures were almost identical to the original target signals simulated using the physics-based model. The VAE was able to capture the complex features in the signals resulting from the interaction of multiple propagating modes in a multi-discontinuous waveguide. The VAE model successfully generated synthetic inspection data by fusing reflections from welds with the reflection from a crack model at specified distances from the transducer on either the right or left side. In some cases, the VAE did not exactly reconstruct the peak amplitude of the reflections. This study demonstrated the potential and highlighted the benefit of using a VAE to generate synthetic data with damage signatures as opposed to using superposition to fuse the damage-free responses containing reflections from welds with a damage signature. The results show that it is possible to generate realistic inspection data for unavailable damage scenarios.