Neutron stars may sustain a non-axisymmetric deformation due to magnetic distortion and are potential sources of continuous gravitational waves (GWs) for ground-based interferometric detectors. With decades of searches using available GW detectors, no evidence of a GW signal from any pulsar has been observed. Progressively stringent upper limits of ellipticity have been placed on Galactic pulsars. In this work, we use the ellipticity inferred from the putative millisecond magnetars in short gamma-ray bursts (SGRBs) to estimate their detectability by current and future GW detectors. For ∼ 1 ms magnetars inferred from the SGRB data, the detection horizon is ∼ 30 Mpc and ∼ 600 Mpc for advanced LIGO (aLIGO) and Einstein Telescope (ET), respectively. Using the ellipticity of SGRB millisecond magnetars as calibration, we estimate the ellipticity and gravitational wave strain of Galactic pulsars and magnetars assuming that the ellipticity is magnetic-distortioninduced. We find that the results are consistent with the null detection results of Galactic pulsars and magnetars with the aLIGO O1. We further predict that the GW signals from these pulsars/magnetars may not be detectable by the currently designed aLIGO detector. The ET detector may be able to detect some relatively low frequency signals (< 50 Hz) from some of these pulsars. Limited by its design sensitivity, the eLISA detector seems not suitable for detecting the signals from Galactic pulsars and magnetars.