Liver cancer has significantly grown in recent years, and thus its mortality rate has also increased since its symptoms appear to be in malignant stages and the treatment path at this stage is extremely challenging. New therapies based on producing heat in cancerous tissues have opened up a new way to treat cancer. This study investigated the treatment of liver cancer by the magnetic hyperthermia approach and nanoparticles (NPs) such as iron oxide (Fe3O4) core with gold (Au), silver (Ag) alloy shell, and molybdenum disulfide (MoS2) coating. The optical properties of these NPs within the tumor, including the extinction coefficient and surface plasmon peak (SPR) as a function of size, structure, different compositions, and thickness, were also examined using the effective medium theory, followed by assessing the impact of temperature distribution through the analytical modeling of an alternating current magnetic field. The results demonstrated that NPs with a compound of Fe3O4-Au_0.25, Ag_0.75@MoS2, a 3 nm thick cover of Au-Ag alloy, and two layers of MoS2 have the best coefficient of extinction and SPR in the biological window. The Au-Ag alloy improved the extinction coefficient and simultaneously prevented the accumulation of magnetic NPs. Considering that the Au-Ag alloy alone cannot function within the range of biological windows, MoS2 was used, which increased the extinction efficiency at higher wavelengths. The examination of the temperature distribution in the tumor for the proposed alloy compound indicated that after a short time from irradiation initiation, the tumor temperature reaches 45 °C. Further, the temperature distribution within the tumor tissue reached its maximum value at the center of the tumor and decreased dramatically while getting away from the center. Finally, the use of magnetic hyperthermia enabled localized delivery of therapeutic doses to malignant tumors, thereby representing superior performance and efficiency over the photothermal method.