Extracting laver aquaculture areas from remote sensing images is very important for laver aquaculture monitoring and scientific management. However, due to the large differences in spectral features of laver aquaculture areas caused by factors such as different growth stages and harvesting conditions, traditional machine learning and deep learning methods face great challenges in achieving accurate and complete extraction of raft laver aquaculture areas. In this article, a reverse attention dual-stream network (RADNet) is proposed for the extraction of laver aquaculture areas with weak spectral responses by comprehensively considering both the aquaculture boundary and surrounding sea background information. RADNet consists of a boundary stream and a segmentation stream. Considering the weaker spectral responses of certain laver aquaculture areas, we introduce a reverse attention module in the segmentation stream to amplify the weaker responses of inapparent laver aquaculture areas. To suppress the response of nonboundary details in the boundary stream, we design a boundary attention module, which is guided by high-level semantics from the segmentation stream. The structural information of the laver aquaculture area learned from the boundary stream will be fed back to the segmentation stream through a specially designed boundary guidance module. The study is conducted in Haizhou Bay, China, and is verified using a self-labeled GF-1 multispectral dataset. The experimental results show that RADNet model performs better in extracting inapparent laver aquaculture areas compared to SOTA models.