We present a photodetector capable of detecting both optical and x-ray picosecond pulses, based on our in-house grown cadmium magnesium telluride (Cd,Mg)Te single crystals. We focused on a specific Cd0.97Mg0.03Te, In-doped crystal composition, because of its bandgap suitable for 800-nm-wavelength light detection and a single-picosecond optical photoresponse. The detector was fabricated as a planar metal-semiconductor-metal structure with interdigitated electrodes and exhibited a linear, Schottky-free, current-voltage characteristics with <40-pA dark current and up to 20-mA/W responsivity. The detector temporal resolution was measured to be ~200 ps full-width-at-half-maximum transient, in response of ~100-fs-wide pulses consisting of either optical (800 nm wavelength) or X-ray (4.5 keV) photons and was limited by the detector housing and 15-GHz bandwidth of the readout oscilloscope. The latter demonstrates the detector is suitable for coarse timing in X-ray free-electron laser/optical femtosecond pump-probe spectroscopy applications. We also demonstrated that due to its very high stopping power, the Cd0.97Mg0.03Te detector responded well to various nuclear gamma sources with energy ranging from 59.6 keV to 660 keV.