SUMMARY Regulating reserve is generally deployed to support system frequency. Because the moment‐to‐moment load fluctuations require continuous frequency control, allocating the reserve for this kind should be flexible to support the function of automatic generation control. Because fast response units not only are assigned to support automatic generation control but also are utilized to deal with credible frequency events, the regulating reserve may be planned as a portion of contingency reserve for security concern, especially for an isolated power system. Without inter‐tie to mutually support system frequency, frequency of an isolated system is much more sensitive to the disturbances than that in the interconnection. Therefore, the scheduling of regulating reserve for an isolated system should be more cautious to take both security and economy into account. This paper describes a methodology that adaptively schedules the regulating reserve based on the dynamic security margin as well as the ongoing demand of the system. Compared with the conventional way of using the historical data, the proposed scheduling method is proven to be more capable of dealing with the change of system condition and furthermore potentially reducing the extraneous reserve. More operational cases will be presented to show the merits of the proposed method that is applied to the ancillary service market. Copyright © 2011 John Wiley & Sons, Ltd.