Various core memory devices have been proposed for utilization in future in‐memory computing technology featuring high energy efficiency. Flash memory is considered as a viable choice owing to its high integration density, stability, and reliability, which has been verified by commercialized products. However, its high operating voltage and slow operation speed issues caused by the tunneling mechanism make its adoption in in‐memory computing applications difficult. In this paper, we introduce a dual‐mode memory device named “ferro‐floating memory”, fabricated using van der Waals (vdW) materials (h‐BN, MoS2, and α‐In2Se3). The vdW material, α‐In2Se3, acts as a polarization control layer for the ferroelectric memory operation and charge storage layer for the conventional flash memory operation. Compared to the tunneling‐based memory operation, the ferro‐floating memory operates 1.9 and 3.3 times faster at 6.7 and 5.8 times lower operating voltages for programming and erasing operations, respectively. The dual‐mode operation improves the linearity of conductance change by 5 times and the dynamic range by 48% through achieving conductance variation regions. Furthermore, we assess the effects of the variation in device operating voltage on neural networks and suggest a memory array operating scheme for maximizing the networks' performance through various training/inference simulations.