Nonlinearly generated broadband ultrafast laser have been increasingly utilized in many applications. However, traditional techniques of characterizing these sources lack the ability to observe the instantaneous features and transitory behaviours of both amplitude and phase. With the advent of the optical time stretch techniques, the instantaneous shotto-shot spectral intensity can be directly measured continuously at an unprecedentedly high speed. Meanwhile, the information of the real-time phase variation, which is carried by the frequency-time mapped spectral signal has yet been fully explored. We present a technique of experimentally measuring the spectral coherence dynamics of broadband pulsed sources. Our method relies on a delayed Young's type interferometer combined with optical time-stretch. We perform the proof-of-principle demonstrations of spectral coherence dynamics measurement on two sources: a supercontinuum source and a fiber ring buffered cavity source, both with a repetition rate of MHz. By employing the optical time stretch with a dispersive fiber, we directly map the spectral interference fringes of the delayed neighbouring pulses and obtain a sufficiently large ensemble of spectral interferograms with a real-time oscilloscope (80Gb/s sampling rate). This enables us to directly quantify the spectral coherence dynamics of the ultrafast sources with a temporal resolution down to microseconds. Having the ensemble of single-shot interferograms, we also further calculate the cross spectral coherence correlation matrices of these ultrafast sources. We anticipate that our technique provides a general approach for experimentally evaluating the spectral coherence dynamics of ultrafast laser generated by the nonlinear processes e.g. modulation instability, supercontinuum generation, and Kerr resonator.