A gradient boosting approach to the Kaggle load forecasting competition

Abstract

We describe and analyse the approach used by Team TinTin (Souhaib Ben Taieb and Rob J Hyndman) in the Load Forecasting track of the Kaggle Global Energy Forecasting Competition 2012. The competition involved a hierarchical load forecasting problem for a US utility with 20 geographical zones. The available data consisted of the hourly loads for the 20 zones and hourly temperatures from 11 weather stations, for four and a half years. For each zone, the hourly electricity load for nine different weeks needed to be predicted without having the location of zones or stations. We used separate models for each hourly period, with component-wise gradient boosting to estimate each model using univariate penalised regression splines as base learners. The models allow for the electricity demand to change with time-of-year, day-of-week, time-of-day, and on public holidays, with the main predictors being current and past temperatures as well as past demand. Team TinTin ranked fifth out of 105 participating teams.

The Fundamentals of Policy Crowdsourcing

“…In TC, organizations post their problems to IT-mediated crowds on platforms such as Innocentive, Eyeka, and Kaggle (Afuah & Tucci 2012) or through in-house platforms such as Challenge.gov (Brabham 2013b). These platforms generally attract and maintain more or less specialized crowds premised on the platform's specific focus; for example, Eyeka's crowd creates advertising collateral for brands, while the crowd at Kaggle focuses on data science solutions (Ben Taieb & Hyndman 2013;Roth & Kimani 2013). When applied to innovation, these platforms have been termed open innovation platforms (Sawhney et al 2003), and represent both the idea generation and problem solving aspects of crowdsourcing (Brabham 2008;Morgan & Wang 2010).…”

Modeling public holidays in load forecasting: a German case study

“…In a small daily load forecasting study, reference [5] compare four different methods to deal with public holidays were evaluated in a neural net framework. First, an additional holiday dummy (A1), second the holidays were treated as a Sunday (A2), third an additional factor variable is introduced (similarly as in [52]) was used to code the day before the holiday, the holiday itself, and the day after the holiday (A3) and fourth a methodology (A2) and (A3) combined. Reference [5] concludes that the method A3 performs best.…”

“…Most notable these are fuzzy linear regression approaches [10,49,50], such as factor variable approches as used in [51,52].…”

A Short-Term User Load Forecasting with Missing Data

“…In the latest competitions of energy forecasting, most of team entries tended to employ ensemble techniques, such as gradient boosting machines (GBMs) and quantile regression forests (QRFs), using more input features [4,5]. A multi-step forecasting strategy with component-wise gradient boosting [6] was used. In addition, nonparametric approaches had been used to do probabilistic forecasting problem, such as Gaussian process [7].…”