When decisions are made in the presence of large-scale stochastic data, it is common to pay more attention to the easy-to-see statistics (e.g., mean) instead of the underlying correlations. One reason is that it is often much easier to solve a stochastic optimization problem by assuming independence across the random data. In this paper, we study the possible loss incurred by ignoring these correlations through a distributionally-robust stochastic programming model, and propose a new concept called Price of Correlations (POC) to quantify that loss. We show that the POC has a small upper bound for a wide class of cost functions, including uncapacitated facility location, Steiner tree and submodular functions, suggesting that the intuitive approach of assuming independent distribution may actually work well for these stochastic optimization problems. On the other hand, we demonstrate that for some cost functions, POC can be particularly large, e.g., the supermodular functions. We propose alternative ways to solve the corresponding distributionally robust models for these functions. As a byproduct, our analysis yields new results on social welfare maximization and the existence of Walrasian equilibria, which may be of independent interest.