Composite ceramic fibers comprising about 80 wt% boron nitride (h-BN) and 20 wt% Si3N4 were fabricated through melt-spinning, electron-beam curing, and pyrolysis up to 1600 °C in atmospheres of NH3 and N2, using a mixture of poly[tri(methylamino)borazine] (PBN) and polysilazane (PSZ). By analyzing the microstructure and composition of the pyrolyzed ceramic fibers, we found the formation of binary phases including crystalline h-BN and amorphous Si3N4. Further investigations confirmed that this heterogeneous microstructure can only be formed when the introduced ratio of Si3N4 is below 30% in mass. The mean modulus and tensile strength of the fabricated composite fibers were about 90 GPa and 1040 MPa, twice the average of the pure h-BN fiber. The dielectric constant and dielectric loss tangent of the composite fibers is 3.06 and 2.94 × 10−3.