Demand response (DR) programs encourage consumers to adapt the time of using electricity based on certain factors, such as cost of electricity, renewable energy availability, and ancillary request. It is one of the most economical methods to improve power system stability and energy efficiency. Residential electricity consumption occupies approximately one-third of global electricity usage and has great potential in DR applications. In this study, we propose a multi-agent optimization approach to incorporate residential DR flexibility into the power system and electricity market. The agents collectively optimize their own interests; meanwhile, the global optimal solution is achieved. The agent perceives its environment, predicts electricity consumption, and forecasts electricity price, based on which it takes intelligent actions to minimize electrical energy cost and time delay of using household appliances. The decision-making action is formulated into a convex program (CP) model. A distributed heuristic algorithm is developed to solve the proposed multi-agent optimization model. Case studies and numerical analysis show promising results with low variation of the aggregated load profile and reduction of electrical energy cost. The proposed approaches can be utilized to investigate various emerging technologies and DR strategies.