Transformation of conventional energy systems into smart grids enables the integration of residential buildings with distributed generations, electro-thermal storages and demand response policies. Further, it improves the household comfort level and helps to preserve the ecological system. Keeping in view the techno-financial impact of residential energy management, optimal handling of residential customers may prove meaningful for peak load reduction, valley filling, and energy conservation. In this regard, this paper devises a residential energy management system (EMS) to optimally schedule appliances, energy sources and electro-thermal storage for reduction of consumption cost and greenhouse (GHG) emissions. Building integrates national grid, natural gas network and solar energy as input carriers, whereas, electricity, heat and cooling as output carriers. To resolve the risk of loss of load, conditional value at risk (CVaR) has been incorporated in the objective function. Comparison demonstrates that under risk-averse approach, energy retaining capability of electric vehicle and thermal energy storage increases by 28.56% and 53.34%, respectively. This stored energy acts as a reserve during absence of solar irradiance and outages on electric and natural gas networks. Moreover, to make EMS more efficient in tracking optimum solutions with faster convergence speed, a hybrid algorithm has been devised by concatenating the modified flower pollination algorithm with mixed-integer linear programming. The proposed algorithm has been validated by comparing its results with the Salp Swarm Algorithm, Grasshopper Optimization Algorithm, Polar Bear Algorithm, Coyote Optimization and Two Cored Flower Pollination Algorithm (FPA). Results manifest that the cost, GHG emissions and execution time drop by 8.98%, 10.81% and 35.064%, respectively. INDEX TERMS Demand response, energy management, residential buildings, smart grids, stochastic model.