Nowadays, there is an increased interest in coffee derivatives (green beans, roasted beans, and coffee by-products (Cascara and Silverskin)) due to their particular chemical composition. This study aimed to compare the content of dry matter, total fat, fatty acids, and fiber (ADF, NDF) of coffee by-products (Cascara and Silverskin) and coffee beans (green and roasted under different conditions). Coffee beans and their by-products were obtained from 100% C. arabica coffee cherries from Panama by dry process. The lowest concentrations of fat corresponded to Cascara 4.24 g·kg−1 and Silverskin 23.70 g·kg−1, respectively. The major fatty acids detected in all samples were palmitic, stearic, oleic, and linoleic acids, the latter two being essential fatty acids. LDA showed that 89.01% of the variability between beans and by-products was explained by lignoceric, myristic, behenic, tricosanoic, arachidic, and heneicosanoic acids. Silverskin appeared to be a good source of lignoceric, myristic, and behenic acids and had a higher concentration of dietary fiber (314.95 g·kg−1) than Cascara (160.03 g·kg−1). Coffee by-products (Silverskin and Cascara) are low-fat products enriched in dietary fiber. Their incorporation, after adjustment, into the global diet may contribute to nutrition security, the sustainability of the coffee sector, and human health.