As a unitary quantum walk with infinitely many internal degrees of freedom, the quantum walk in terms of quantum Bernoulli noise (recently introduced by Wang and Ye) shows a rather classical asymptotic behavior, which is quite different from the case of the usual quantum walks with a finite number of internal degrees of freedom. In this paper, we further examine the structure of the walk. By using the Fourier transform on the state space of the walk, we obtain a formula that links the moments of the walk's probability distributions directly with annihilation and creation operators on Bernoulli functionals. We also prove some other results on the structure of the walk. Finally, as an application of these results, we establish a quantum central limit theorem for the annihilation and creation operators themselves.