Fruits and vegetables are an important supplier of biological active substances, such as vitamins and secondary metabolites (SM) for human nutrition, but also for further industrial applications. Pepper (Capsicum annuum) is grown and consumed all over the world as a fresh vegetable or dried as a spice. It is also used as a coloring agent, as well for medical purposes. Pepper fruits are considered as an attractive source of health-related compounds, but other organs like the leaves and stem also contain considerable amounts of antioxidants, e.g., phenolic compounds. This indicates potential for valorization of residual biomass from horticultural production by using innovative bioeconomic concepts. Herein, we present an overview about the biosynthesis of phenolic compounds, with a special focus on flavonoids and their regulation in pepper, the current knowledge of amounts and distribution of these valuable substances, as well as possible strategies for: (1) increasing flavonoid contents in pepper, (2) improving the nutritional value of fruits, and (3) new concepts for utilization of residual biomass from horticultural production.