According to the Standard Model (SM), we expect to find a proton for each decaying neutron. However, the experiments counting the number of decayed neutrons and produced protons disagree. This discrepancy suggests that neutrons might have an exotic decay channel to Dark Sector (DS) particles. In this paper, we explore a scenario where neutrons decay to a dark Dirac fermion χ and a non-Abelian dark gauge boson W 0. In our proposed scenario, the DS has three portals with the SM sector: (1) the fermion portal coming from the mixing of the neutron with χ, (2) a scalar portal, and (3) a nonrenormalizable kinetic mixing between photon and dark gauge bosons which induces a vector portal between the two sectors. We discuss the cosmological implications of this scenario assuming DS particles are produced via freeze-in. The fermion and the scalar portal leads to the overproduction of DS particles by the time of the Cosmic Microwave Background (CMB), and thus we disable these two portals in the early universe. For that, we require the maximum temperature of the universe to be lower than m χ. We rely on the vector portal to connect the two sectors in the early universe, and we discuss the phenomenological bounds on the model. The main constraints come from the Big Bang Nucleosynthesis, ensuring the right relic abundance of dark matter, and the observation of large neutron stars.