We propose a framework to address the observed baryon asymmetry considering neutrinos as Dirac particles and dark matter to be self-interacting. The Standard Model is extended by U(1)B−L × U(1)D symmetry. In addition to the three right-handed neutrinos, the Standard Model particle content is extended by two more fermion fields; one of them is a S U(2) singlet and the other one is S U(2) doublet, both charged under the extended symmetry. A Z2 symmetry is imposed under which the doublet is positive while the singlet is negative. The singlet, being the lightest dark sector particle, acts as the dark matter. The U(1)D symmetry of the dark sector is spontaneously broken and the corresponding gauge boson Z′ not only mediates the self-interaction among dark matter particles, but also facilitates the annihilation of the symmetric component of dark matter. CP-violating out of equilibrium decay of another heavy Z2 odd S U(2) scalar doublet N into leptonic and dark sectors produce the observed baryon asymmetry and dark matter density. Moreover, Z′ also mixes with the standard model Z-boson opening up a portal for the direct detection of dark matter.