Penicillium digitatum is the main postharvest pathogen of citrus fruit. Although the inner fruit peel part (albedo) is less resistant than the outer part (flavedo) to P. digitatum, the global mechanisms involved in their different susceptibility remain unknown. Here, we examine transcriptome differences between both tissues at fruit harvest and in their early responses to infection. At harvest, not only was secondary metabolism, involving phenylpropanoids, waxes, and terpenoids, generally induced in flavedo vs. albedo, but also energy metabolism, transcription factors (TFs), and biotic stress-related hormones and proteins too. Flavedo-specific induced responses to infection might be regulated in part by ERF1 TF, and are related to structural plant cell wall reinforcement. Other induced responses may be related to H2O2, the synthesis of phenylpropanoids, and the stress-related proteins required to maintain basal defense responses against virulent pathogens, whereas P. digitatum represses some hydrolase-encoding genes that play different functions and auxin-responsive genes in this peel tissue. In infected albedo, the repression of transport and signal transduction prevail, as does the induction of not only the processes related to the synthesis of flavonoids, indole glucosinolates, cutin, and oxylipins, but also the specific genes that elicit plant immunity against pathogens.