Bakery products are usually formulated with solid fats, like margarines and shortenings, which contain high levels of saturated and trans-fatty acids and have negative effects on human health. In this study, hydroxypropyl methylcellulose (HPMC) and xanthan gum (XG) were used as oleogelators to prepare oleogels, using sunflower and olive oil, as substitutes for margarine in baked or steamed buns. The effect of oleogels on the physical properties of the buns was evaluated by analyzing the crumb structure, specific volume, height, and texture. In addition, a triangular discriminatory sensory test was conducted, and lipid digestibility was assessed through in vitro digestion studies. Replacement of margarine with oleogels produced steamed buns with no differences in the crumb structure, volume, height, and texture; however, in baked buns, a less porous and harder structure was produced. No differences in texture were observed between the margarine buns and buns made with oleogels when the triangular test was conducted. The extent of lipolysis was not affected when margarine was replaced by oleogels in the baked and steamed buns. The results suggest that using oleogels instead of margarine in buns could represent an interesting strategy to prepare healthier bakery products.