Coherent errors, which arise from collective couplings, are a dominant form of noise in many realistic quantum systems, and are more damaging than oft considered stochastic errors. Here, we propose integrating stabilizer codes with coherent-error-avoiding codes by code concatenation. Namely, by concatenating an [[n, k, d]] stabilizer outer code with dual-rail inner codes, we obtain a [[2n, k, d]] non-stabilizer constant-excitation code immune from coherent phase errors and also equivalent to a Pauli-rotated stabilizer code. When the stabilizer outer code is fault-tolerant, the constant-excitation code has a positive fault-tolerant threshold against stochastic errors. Setting the outer code as a four-qubit amplitude damping code yields an eight-qubit constant-excitation code that corrects a single amplitude damping error, and we analyze this code's potential as a quantum memory. We numerically demonstrate that fault-tolerant quantum error correction overheads can be significantly reduced the noise is dominated by coherent phase errors with some stochastic errors.