BackgroundPositron emission tomography (PET) and computed tomography (CT) play a vital role in tumor‐related medical diagnosis, assessment, and treatment planning. However, full‐dose PET and CT pose the risk of excessive radiation exposure to patients, whereas low‐dose images compromise image quality, impacting subsequent tumor recognition and disease diagnosis.PurposeTo solve such problems, we propose a Noise‐Assisted Hybrid Attention Network (NAHANet) to reconstruct full‐dose PET and CT images from low‐dose PET (LDPET) and CT (LDCT) images to reduce patient radiation risks while ensuring the performance of subsequent tumor recognition.MethodsNAHANet contains two branches: the noise feature prediction branch (NFPB) and the cascaded reconstruction branch. Among them, NFPB providing noise features for the cascade reconstruction branch. The cascaded reconstruction branch comprises a shallow feature extraction module and a reconstruction module which contains a series of cascaded noise feature fusion blocks (NFFBs). Among these, the NFFB fuses the features extracted from low‐dose images with the noise features obtained by NFPB to improve the feature extraction capability. To validate the effectiveness of the NAHANet method, we performed experiments using two public available datasets: the Ultra‐low Dose PET Imaging Challenge dataset and Low Dose CT Grand Challenge dataset.ResultsAs a result, the proposed NAHANet achieved higher performance on common indicators. For example, on the CT dataset, the PSNR and SSIM indicators were improved by 4.1 dB and 0.06 respectively, and the rMSE indicator was reduced by 5.46 compared with the LDCT; on the PET dataset, the PSNR and SSIM was improved by 3.37 dB and 0.02, and the rMSE was reduced by 9.04 compared with the LDPET.ConclusionsThis paper proposes a transformer‐based denoising algorithm, which utilizes hybrid attention to extract high‐level features of low dose images and fuses noise features to optimize the denoising performance of the network, achieving good performance improvements on low‐dose CT and PET datasets.