We systematically develop theories of critical phenomena with memory of a power-law decaying long-range temporal interaction parameterized by a constant $\theta>0$ for a space dimension $d$ both below and above an upper critical dimension $d_c=6-2/\theta$. We first provide more evidences to confirm the previous theory that a dimensional constant $\mathfrak{d}_t$ is demanded to rectify a hyperscaling law among others. Next, for $d<d_c$, we develop a renormalization-group theory to the leading nontrivial order explicitly and to higher orders formally in $\epsilon=d_c-d$ but to zero order in $\varepsilon=\theta-1$ and find that more scaling laws besides the hyperscaling law are broken due to the breaking of the fluctuation-dissipation theorem. Moreover, because dynamics and statics are intimately interwoven, even the static critical exponents involve contributions from the dynamics and hence do not restore the short-range exponents even for $\theta=1$ and the crossover between the short-range and long-range fixed points is discontinuous contrary to the case of long-range spatial interaction. In addition, a new scaling law relating the dynamic critical exponent with the static ones emerges. However, once $\mathfrak{d}_t$ is displaced by a series of $\epsilon$ and $\varepsilon$ such that most values of the critical exponents are changed, all scaling laws are saved again, even though the fluctuation-dissipation theorem keeps violating. Then, for $d\ge d_c$, we develop an effective-dimension theory by carefully discriminating the corrections of both temporal and spatial dimensions and find three different regions with unique properties. All these results show that the dimensional constant $\mathfrak{d}_t$ is the fundamental ingredient of the theories for critical phenomena with memory. However, its value continuously varies with the space dimension and vanishes exactly at $d=4$, reflecting the variation of the amount of the temporal dimension that is transferred to the spatial one with the strength of fluctuations. Moreover, special finite-size scaling ubiquitously appears except for $d=d_{c0}$