All‐optical and fully reconfigurable transmissive diffractive optical neural network (DONN) architectures emerge as high‐throughput and energy‐efficient machine learning (ML) hardware accelerators in broad applications. However, current device and system implementations have limited performance. In this work, a novel transmissive diffractive device architecture, a digitized phase‐change material (PCM) heterostack, which consists of multiple nonvolatile PCM layers with different thicknesses, is demonstrated. Through this architecture, the advantages of PCM electrical and optical properties can be leveraged and challenges associated with multilevel operations in a single PCM layer can be mitigated. Through proof‐of‐concept experiments, the electrical tuning of one PCM layer is demonstrated in a transmissive spatial light modulation device, and thermal analysis guides the design of multilayer devices and DONN systems to avoid thermal cross talk if individual heterostacks are assembled into an array. Further, a heterostack containing three PCM layers is designed based on experimental results to produce a large‐phase modulation range and uniform coverage, and the ML performance of DONN systems with the designed heterostack is evaluated. The developed device architecture is practically feasible and scalable for future energy‐efficient, fast‐reconfigured, and compact transmissive DONN systems.