Designing strategies for an effective transformation of food waste into high-value products is a priority to address environmental sustainability concerns. Coffee silverskin is the major by-product of the coffee roasting industry, being rich in compounds with health benefits. Such composition gives it the potential to be transformed into high-value products. In this study, coffee silverskin extracts were enriched, regarding caffeine and chlorogenic acid contents, by adsorbent column chromatography. The compounds content increased 3.08- and 2.75-fold, respectively, compared to the original extract. The enriched fractions were loaded into nano-phytosomes or cholesterol-incorporated nano-phytosomes (first coating layers) to improve the physiochemical properties and permeation rate. These nano-lipid carriers were also subjected to a secondary coating with different natural polymers to improve protection and stability against degradation. In parallel, and for comparison, different natural polymers were also used as first coating layers. The produced particles were evaluated regarding product yield, encapsulation efficiency, loading capacity, particle size, surface charge, and in vitro release simulating gastrointestinal conditions. All samples exhibited anionic surface charge. FTIR and molecular docking confirmed interactions between the phytoconstituents and lipid bilayers. The best docking score was observed for 5-caffeoylquinic acid (chlorogenic acid) exhibiting a stronger hydrogen binding to the lipid bilayer. Among several kinetic models tested, the particle release mechanism fitted well with the First-order, Korsmeyer–Peppas, and Higuchi models. Moreover, most of the formulated particles followed the diffusion-Fick law and anomalous transport.