California forests have recently experienced record breaking wildfires and tree mortality from droughts, However, there is inadequate monitoring, and limited data to inform policies and management strategies across the state. Although forest surveys and satellite observations of forest cover changes exist at medium to coarse resolutions (30–500 m) annually, they remain less effective in mapping small disturbances of forest patches (<5 m) occurring multiple times a year. We introduce a novel method of tracking California forest cover using a supervised U-Net deep learning architecture and PlanetScope’s Visual dataset which provides 3-band RGB (Red, Green, and Blue) mosaicked imagery. We created labels of forest and non-forest to train the U-Net model to map tree cover based on a semi-unsupervised classification method. We then detected changes of tree cover and disturbance with the U-Net model, achieving an overall accuracy of 98.97% over training data set, and 95.5% over an independent validation dataset, obtaining a precision of 82%, and a recall of 74%. With the predicted tree cover mask, we created wall to wall monthly tree cover maps over California at 4.77 m resolution for 2020, 2021, and 2022. These maps were then aggregated in a post-processing step to develop annual maps of disturbance, while accounting for the time of disturbance and other confounding factors such as topography, phenological and snow cover variability. We compared our high-resolution disturbance maps with wildfire GIS survey data from CALFIRE, and satellite-based forest cover changes and achieved an F-1 score of 54% and 88% respectively. The results suggest that high-resolution maps capture variability of forest disturbance and fire that wildfire surveys and medium resolution satellite products cannot. From 2020 to 2021, California maintained 30,923.5 sq km of forest while 5,994.9 sq km were disturbed. The highest observed forest loss rate was located at the Sierra Nevada mountains at 21.4% of the forested area being disturbed between 2020 and 2021. Our findings highlight the strong potential of deep learning and high-resolution RGB optical imagery for mapping complex forest ecosystems and their changes across California, as well as the application of these techniques on a national to global scale.