Heuristic and Backtracking Algorithms for Improving the Performance of Efficiency Analysis Trees

Abstract

In the literature of Economics, Engineering and Operations Research, the estimation of production frontiers is a current hot topic. Many parametric and nonparametric methodologies have been introduced for estimating technical efficiency of a set of units (for example, firms) from the production frontier. However, few of these methodologies are based upon machine learning techniques, despite being a rising field of research. Recently, a bridge has been built between these literatures, machine learning and production theory, through a new technique proposed in Esteve et al (2020), called Efficiency Analysis Trees (EAT). The algorithm developed from EAT, based on the well-known Classification and Regression Trees (CART) machine learning technique, is a greedy technique that uses a particular heuristic for the selection of the next node to be split during the decision tree development process. Nevertheless, as we show in this paper, for different sample sizes and number of variables, the heuristic used by EAT is not capable of obtaining the tree with the minimum mean square error (MSE). For this reason, in this paper, a backtracking technique is implemented to improve the MSE obtained by the EAT algorithm. Additionally, a pair of new algorithms are introduced which combine the heuristic technique used by the standard EAT and the backtracking algorithm to enhance the reduction of the MSE, while decreasing the computation time. Our research is based on some simulated experiments. According to our computational results, the combination of the heuristic and the backtracking algorithm, in particular, that in which the tree growth starts with heuristics and ends with backtracking, has achieved an accuracy similar to that of backtracking and within a reasonable computational time. The contribution of the paper could be of special interest for industrial engineers interested in measuring efficiency and productivity of industrial processes in many sectors, such as energy, agri-food or service industries.

Efficiency analysis trees as a tool to analyze the quality of university education

“…However, the above is reachable because the tree's growth occurs evolutionarily. Besides, the fitting process is achieved by minimizing the MSE and using stopping rules linked to the size of the database, avoiding empty leaf nodes [25]. For its part, it is necessary to mention that the importance of the tool lies in being useful for decision-making in educational environments.…”

Assesing the influence of environmental variables on the performance of water companies: An efficiency analysis tree approach

“…where denotes the sub-tree that is produced from applying the EAT algorithm, is 𝑇 𝑘 the number of splits, , and presents the highest al., 2020). Thus, the predictor function is non-decreasing, and the estimated production frontier looks like a step function (Aparicio et al, 2021;Esteve et al, 2021a). Thus, the production technology estimated using EAT is defined as follows:…”

“…However, it is challenging to select the number of bootstrap replications and optimal number of m (Villegas et al, 2019). Consequently, Esteve et al (2020Esteve et al ( , 2021a developed a new technique, called Efficiency Analysis Trees (EAT). This technique combines the Classification and Regression Trees (CART) proposed by Breiman et al (1984) with non-parametric analysis to measure efficiency.…”

“…As a result, the estimated frontier utilizes a step function that allows efficiency scores to be measured. Esteve et al (2020Esteve et al ( , 2021a Thus, this study aimed to evaluate the efficiency of water utilities using EAT, the newly developed technique. The regression tree allowed the maximum (frontier) expenditure required to provide water services to be visualised at different thresholds (rules).…”

A study of algorithms for solving nonlinear two-level programming problems oriented to decision tree models

“…CART is a binary tree with a binary cut, which cuts the data into two copies at a time, into the left and right subtrees, and there are two of each non-leaf node. CART has more leaf nodes than non-leaves, so it can be used for both classification and can be used for regression [17]. The expression form is as follows:…”