In this paper, we study online impression allocation in display advertising with budgeted advertisers. That is, upon arrival of each impression, cost and revenue vectors are revealed and the impression should be assigned to an advertiser almost immediately. Without any assumption on the distribution/arrival of impressions, we propose a framework to capture the risk to the ad network for each possible allocation; impressions are allocated to advertisers such that the risk of ad network is minimized. In practice, this translates to starting with an initial estimate of dual prices and updating them according to the belief of the ad network toward the future demand and remaining budgets. We apply our algorithms to a real data set, and we empirically show that Kullback-Leibler divergence risk measure has the best performance in terms of revenue and balanced budget delivery.