Visualising forecasting algorithm performance using time series instance spaces

Kang, Yanfei; Hyndman, Rob J.; Smith‐Miles, Kate

doi:10.1016/j.ijforecast.2016.09.004

Cited by 140 publications

(112 citation statements)

References 22 publications

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…The meta-data, composed of the features and algorithm performance for all the instances in I, is used to learn the mapping g(f(x), y(α, x)) that projects an instance x from a high-dimensional feature space to a two-dimensional space, which we call the instance space. In earlier work, this projection was achieved using principal component analysis, and applied to problems as diverse as graph coloring [21], time series forecasting [7], and software test case generation methods [18]. In this paper, we adopt the latest version of the evolving methodology described in [13], applied to machine learning algorithms, where a customized projection algorithm was developed to obtain an optimal projection that aims to expose linear trends in both features and algorithm performance to aid interpretability.…”

Section: Instance Space Analysismentioning

confidence: 99%

Instance space analysis for a personnel scheduling problem

Kletzander

Musliu

Smith‐Miles

2020

Ann Math Artif Intell

Self Cite

View full text Add to dashboard Cite

This paper considers the Rotating Workforce Scheduling Problem, and shows how the strengths and weaknesses of various solution methods can be understood by the in-depth evaluation offered by a recently developed methodology known as Instance Space Analysis. We first present a set of features aiming to describe hardness of test instances. We create a new, more diverse set of instances based on an initial instance space analysis that reveals gaps in the instance space, and offers the opportunity to generate additional instances to add diversity to the test suite. The results of three algorithms on our extended instance set reveal insights based on this visual methodology. We observe different regions of strength and weakness in the instance space for each algorithm, as well as a phase transition from feasible to infeasible instances with more challenging instances at the phase transition boundary. This rigorous and insightful approach to analyzing algorithm performance highlights the critical role played by the choice of test instances, and the importance of ensuring diversity and unbiasedness of test instances to support valid conclusions.

show abstract

Section: Instance Space Analysismentioning

confidence: 99%

Instance space analysis for a personnel scheduling problem

Kletzander

Musliu

Smith‐Miles

2020

Ann Math Artif Intell

Self Cite

View full text Add to dashboard Cite

show abstract

“…The study of [20] visualized forecasting algorithm performance using time series instance space. They proved that the ARIMA method does the best overall forecasting performance.…”

Section: Related Literaturementioning

confidence: 99%

Applying Data Mining Techniques in Predicting Index and non-Index Crimes

Delima¹

2019

IJMLC

View full text Add to dashboard Cite

An increasing incidence of crime has led to the development and use of computer-aided diagnosis system, tools and methods in analyzing, classifying and predicting crimes. This paper clusters municipalities in Surigao del Norte using K-Means algorithm. This is instrumental in finding identical traits, patterns and values in categorizing municipalities with much, more, and most number of recorded index and non-index crimes from 2013-2017. Prediction of its occurrence for the year 2018-2022 was also provided using ARIMA (1,0,7) model. Results showed that Surigao City has the most number of recorded index and non-index crimes. Followed by the municipality of Placer, Claver, and Dapa of cluster 2 and Del Carmen of cluster 3. Further, physical injury, homicide, violation of special laws, car napping, reckless imprudence resulting to physical injury, and other non-index crimes has 26%, 25%, 25%, 24%, 24%, and 23% forecasted increase for the year 2018-2022 with the highest occurrence in years 2018, 2018, 2020, 2018, 2020, and 2020 respectively. Future researchers may utilize other data mining techniques supported by a better accuracy result.

show abstract

“…The detection of events in noisy data is of particular interest in the case of turbulent atmospheric data sets, especially given the need for more sophisticated forecasting systems (Belušić and Mahrt, 2012;Fulcher, 2018;Gamage and Hagelberg, 1993;Kang et al, 2014Kang et al, , 2017Sun et al, 2015). One of the more common event detection methods leverages the continuous or discrete wavelet transform (Gamage and Hagelberg, 1993;Kumar and Foufoula-Georgiou, 1997;Lilly, 2017).…”

Section: Introductionmentioning

confidence: 99%

Atmospheric condition identification in multivariate data through a metric for total variation

Hamilton

2020

Atmos. Meas. Tech.

View full text Add to dashboard Cite

Abstract. Identification of atmospheric conditions within a multivariable atmospheric data set is a necessary step in the validation of emerging and existing high-fidelity models used to simulate wind plant flows and operation. Atmospheric conditions relevant for wind energy research include stationary conditions, given the need for well-converged statistics for model validation, as well as conditions observed less frequently, such as extreme atmospheric events, which are used in wind turbine and wind plant design. Aggregation of observations without regard to covariance between time series discounts the dynamical nature of the atmosphere and is not sufficiently representative of atmospheric conditions. Identification and characterization of continuous time periods with atmospheric conditions that have a high value for analysis or simulation set the stage for more advanced model validation and the development of real-time control and operational strategies. The current work explores a single metric for variation in a multivariate data sample that quantifies variability within each channel as well as covariance between channels. The total variation is used to identify conditions of interest that conform to desired objective functions, such as stationary conditions, ramps or waves of wind speed, and changes in wind direction. Total variation is somewhat sensitive to the presence of outliers in the input data, and the method is best complemented by quality-control procedures to ensure reliable results. The direct detection and classification of events or conditions of interest within atmospheric data sets is vital to developing our understanding of wind plant response and to the formulation of forecasting and control models.

show abstract

Visualising forecasting algorithm performance using time series instance spaces

Cited by 140 publications

References 22 publications

Instance space analysis for a personnel scheduling problem

Instance space analysis for a personnel scheduling problem

Applying Data Mining Techniques in Predicting Index and non-Index Crimes

Atmospheric condition identification in multivariate data through a metric for total variation

Contact Info

Product

Resources

About