We have investigated the influence of B-site doping on the crystal and magnetic structure in La(0.5)Ca(0.5)Mn(1 - x)B(x)O(3) (B = Fe, Ru, Al and Ga) compounds using neutron diffraction, small angle neutron scattering, magnetization and resistivity techniques. The B-site doped samples are isostructural and possess an orthorhombic structure in the Pnma space group at 300 K. A structural transition from orthorhombic to monoclinic is found to precede the magnetic transition to the CE-type antiferromagnetic state in a few of these samples. On doping with Fe, the charge and orbitally ordered CE-type antiferromagnetic state is suppressed, followed by growth of the ferromagnetic insulating phase in 0.02 ≤ x ≤ 0.06 compounds. At higher Fe doping in x > 0.06, the ferromagnetic state is also suppressed and no evidence of long range magnetic ordering is observed. In Ru doped samples (0.01 ≤ x ≤ 0.05), the ferromagnetic metallic state is favored at T(C)≈200 K and T(MI)≈125 K and no significant change in T(C) and T(MI) as a function of Ru doping is found. In contrast, with non-magnetic Al substitution for 0.01 ≤ x ≤ 0.03, the charge ordered CE-type antiferromagnetic state coexists with the ferromagnetic metallic phase. With further increase in Al doping (0.05 ≤ x ≤ 0.07), both CE-type antiferromagnetic and ferromagnetic phases are gradually suppressed. This behavior is accompanied by the evolution of an A-type antiferromagnetic insulating state. Eventually, at higher Al doping (0.10 ≤ x ≤ 0.13), this phase is also suppressed and the signature of a spin glass like transition is evident in M(T). Likewise, substitution with Ga is observed to induce similar effects to those described for Al doped samples. The presence of short ranged ferromagnetic ordering has been further explored using small angle neutron scattering measurements in a few of the selected samples.