This paper describes leading-edge research to explore and demonstrate new and unique nanoscale magnetic composites for high-frequency RF applications. Passivated cobalt nanoparticles were chemically synthesized and dispersed in epoxy to fabricate nanocomposite thick films. The high permeability comes from enhanced coupling between the metal nanoparticles while the insulating polymer matrix prevents eddy current loss and improves stability with frequency. Test vehicles were fabricated to demonstrate integration of these composites in organic substrates and to characterize the high-frequency properties.The frequency-dependent magnetic properties in 100-500 MHz range were extracted by impedance spectroscopy. Magnetic toroids were mechanically pressed with the metalinsulator powder. By refining the processing, permeability of 2.7 was demonstrated at VHF frequencies. The loss tangent was less than 0.04 at these frequencies.The GHz frequency-dependent material characteristics of the magneto-dielectric films were extracted from cornerprobing of parallel-plate resonators and strip inductors. By engineering the composite structures at nanoscale, a combination of stable permeability of ~2 at 1-5 GHz and permittivity of 7, not previously reported, was achieved with polymer composites for antenna miniaturization. The magnetic nanomaterials with low loss, described in this paper, can benefit several other RF and power components, leading to their miniaturization and performance enhancement in emerging RF sub-systems. The metal composite structures also lead to high permittivity in the GHz frequencies which can benefit such RF components as antennas, by allowing closer impedance matching with air.