We propose a framework to construct conversational Web agents, which guide the visitors of Web sites, from actual dialogues incrementally. We apply the Wizard of Oz method enhanced by adding the functionality of learning dialogue models to the process. In this method, a developer collects annotated dialogues by chatting with users. At first, the developer has to input almost all replies. As the learning proceeds, the system infers proper utterances and the load of developer is reduced. Finally, a conversational agent is constructed. We developed a system to construct such kind of agents in the Web environment. The features of this system are: (a) FSM-based dialogue models and incremental algorithms of learning probabilistic DFAs; (b) using annotations for the meaning of each utterance and for contents associated with Web pages; and (c) a character interface with speech functionality. We also examined how the developer's cost is reduced as the growth of the dialogue models, applying it to Kyoto tour guide task.