Ca3CoMnO6, a quasi-1D Ising chain at low temperature offers rich fundamental physics and applications. We have studied the dielectric and magnetoelectric coupling in Fe doped Ca3CoMnO6 (CCMO) bulk ceramics prepared by co-precipitation technique. Single phase hexagonal crystal structure having R-3C space group was confirmed by x-ray diffraction. Extremely low currents were observed up to 20% Fe doping. Doping dependence of magnetoresistance (MR) revealed both positive and negative MR, with anomalously high MR values beyond 3000% in diluted Fe doped CCMO; whereas the higher doping of Fe was found to result in negative MR due to enhanced magnetostriction effects. The dielectric study was carried out for a range of 20 Hz to 10 MHz. The negative value of the colossal magnetodielectric induced in the Fe doped samples can be attributed to the magnetostriction effect along with interfacial Maxwell-Wagner polarization.