Extreme value statistics (EVS) concerns the study of the statistics of the maximum or the minimum of a set of random variables. This is an important problem for any time-series and has applications in climate, finance, sports, all the way to physics of disordered systems where one is interested in the statistics of the ground state energy. While the EVS of 'uncorrelated' variables are well understood, little is known for strongly correlated random variables. Only recently this subject has gained much importance both in statistical physics and in probability theory. In this review, we will first recall the classical EVS for uncorrelated variables and discuss the three universality classes of extreme value limiting distribution, known as the Gumbel, Fréchet and Weibull distribution. We then show that, for weakly correlated random variables with a finite correlation length/time, the limiting extreme value distribution can still be inferred from that of the uncorrelated variables using a renormalisation group-like argument. Finally, we consider the most interesting examples of strongly correlated variables for which there are very few exact results for the EVS. We discuss few examples of such strongly correlated systems (such as the Brownian motion and the eigenvalues of a random matrix) where some analytical progress can be made. We also discuss other observables related to extremes, such as the density of near-extreme events, time at which an extreme value occurs, order and record statistics, etc.