The optimization of extraction by using solvents of phenolic compounds (TPh) of grape pomace (GP) based on a central composite design was investigated. The GP was characterized, and preliminary assays were conducted with five different solvents (water, ethanol, acetone, methanol, and butanol) and the aqueous mixtures thereof. Ethanol and acetone were revealed to be the best solvents for TPh extraction. The main extraction parameters (temperature—T, time—t, solvent concentration, and liquid–solid ratio—L/S) were optimized by using a central composite design. The optimized conditions for the ethanol extraction (T = 60 °C, t = 1.5 h, L/S = 25 mL/gdryGP) and for acetone (T = 50 °C, t = 1.5 h, L/S = 25 mL/gdryGP) were determined. Single-stage extraction revealed a TPh of 45.18 ± 9.51 mgGAE/gdryGP for acetone and a TPh of 38.70 ± 3.64 mgGAE/gdryGP for ethanol. The characterization of the extracts revealed the presence of gallic acid, caffeic acid, syringic acid, vanillic acid, chlorogenic acid, and p-coumaric acid, where the concentration of the first three compounds stands out in all extracts. A three-stage extraction increased the yield of ethanol to 63.3 mg GAE/gdryGP and the yield of acetone to 59.2 mg GAE/gdryGP. Overall, both solvents allow the extraction of phenolic compounds of grape pomace, but ethanol is commonly considered a greener solvent for this purpose.