Histopathology for tumor margin assessment is time‐consuming and expensive. High‐resolution full‐field optical coherence tomography (FF‐OCT) images fresh tissues rapidly at cellular resolution and potentially facilitates evaluation. Here, we define FF‐OCT features of normal and neoplastic skin lesions in fresh ex vivo tissues and assess its diagnostic accuracy for malignancies. For this, normal and neoplastic tissues were obtained from Mohs surgery, imaged using FF‐OCT, and their features were described. Two expert OCT readers conducted a blinded analysis to evaluate their diagnostic accuracies, using histopathology as the ground truth. A convolutional neural network was built to distinguish and outline normal structures and tumors. Of the 113 tissues imaged, 95 (84%) had a tumor (75 basal cell carcinomas [BCCs] and 17 squamous cell carcinomas [SCCs]). The average reader diagnostic accuracy was 88.1%, with a sensitivity of 93.7%, and a specificity of 58.3%. The artificial intelligence (AI) model achieved a diagnostic accuracy of 87.6 ± 5.9%, sensitivity of 93.2 ± 2.1%, and specificity of 81.2 ± 9.2%. A mean intersection‐over‐union of 60.3 ± 10.1% was achieved when delineating the nodular BCC from normal structures. Limitation of the study was the small sample size for all tumors, especially SCCs. However, based on our preliminary results, we envision FF‐OCT to rapidly image fresh tissues, facilitating surgical margin assessment. AI algorithms can aid in automated tumor detection, enabling widespread adoption of this technique.