A problem faced by many instructors is that of designing exams that accurately assess the abilities of the students. Typically, these exams are prepared several days in advance, and generic question scores are used based on rough approximation of the question difficulty and length. For example, for a recent class taught by the author, there were 30 multiple choice questions worth 3 points, 15 true/false with explanation questions worth 4 points, and 5 analytical exercises worth 10 points. We describe a novel framework where algorithms from machine learning are used to modify the exam question weights in order to optimize the exam scores, using the overall final score as a proxy for a student's true ability. We show that significant error reduction can be obtained by our approach over standard weighting schemes, i.e., for the final and midterm exam, the mean absolute error for prediction decreases by 90.58% and 97.70% for linear regression approach respectively resulting in better estimation. We make several new observations regarding the properties of the "good" and "bad" exam questions that can have impact on the design of improved future evaluation methods.

Algorithms for equilibrium computation generally make no attempt to ensure that the computed strategies are understandable by humans. For instance the strategies for the strongest poker agents are represented as massive binary files. In many situations, we would like to compute strategies that can actually be implemented by humans, who may have computational limitations and may only be able to remember a small number of features or components of the strategies that have been computed. For example, a human poker player or military leader may not have access to large precomputed tables when making real-time strategic decisions. We study poker games where private information distributions can be arbitrary (i.e., players are dealt cards from different distributions, which depicts the phenomenon in large real poker games where at some points in the hand players have different distribution of hand strength by applying Bayes' rule given the history of play in the hand thus far). We create a large training set of game instances and solutions, by randomly selecting the information probabilities, and present algorithms that learn from the training instances to perform well in games with unseen distributions. We are able to conclude several new fundamental rules about poker strategy that can be easily implemented by humans.

In many settings people must give numerical scores to entities from a small discrete set. For instance, rating physical attractiveness from 1-5 on dating sites, or papers from 1-10 for conference reviewing.We study the problem of understanding when using a different number of options is optimal. We consider the case when scores are uniform random and Gaussian. We study computationally when using 2, 3, 4, 5, and 10 options out of a total of 100 is optimal in these models (though our theoretical analysis is for a more general setting with k choices from n total options as well as a continuous underlying space). One may expect that using more options would always improve performance in this model, but we show that this is not necessarily the case, and that using fewer choices-even just two-can surprisingly be optimal in certain situations. While in theory for this setting it would be optimal to use all 100 options, in practice this is prohibitive, and it is preferable to utilize a smaller number of options due to humans' limited computational resources. Our results could have many potential applications, as settings requiring entities to be ranked by humans are ubiquitous. There could also be applications to other fields such as signal or image processing where input values from a large set must be mapped to output values in a smaller set.

This paper illustrates the application of machine learning algorithms in predictive analytics for local governments using administrative data. The developed and tested machine learning predictive algorithm overcomes known limitations of the conventional ordinary least squares method. Such limitations include but not limited to imposed linearity, presumed causality with independent variables as presumed causes and dependent variables as presume result, likely high multicollinearity among features, and spatial autocorrelation. The study applies the algorithms to 311 non-emergency service requests in the context of Miami-Dade County. The algorithms are applied to predict the volume of 311 service requests and the community characteristics affecting the volume across Census tract neighborhoods. Four common families of algorithms and an ensemble of them are applied. They are random forest, support vector machines, lasso and elastic-net regularized generalized linear models, and extreme gradient boosting. Two feature selection methods, namely Boruta and fscaret, are applied to identify the significant community characteristics. The results show that the machine learning algorithms capture spatial autocorrelation and clustering. The features generated by fscaret algorithms are parsimonious in predicting the 311 service request volume.

No abstract