Analysing ProB’s Constraint Solving Backends

Abstract

We evaluate the strengths and weaknesses of different backends of the ProB constraint solver. For this, we train a random forest over a database of constraints to classify whether a backend is able to find a solution within a given amount of time or answers unknown. The forest is then analysed in regards of feature importances to determine subsets of the B language in which the respective backends excel or lack for performance. The results are compared to our initial assumptions over each backend's performance in these subsets based on personal experiences. While we do employ classifiers, we do not aim for a good predictor, but are rather interested in analysis of the classifier's learned knowledge over the utilised B constraints. The aim is to strengthen our knowledge of the different tools at hand by finding subsets of the B language in which a backend performs better than others.

Formal Group Fairness and Accuracy in Automated Decision Making

“…Supervised learning models are trained on past data to recognize inlaying patterns and to make future predictions [22]. Based on these models, final decisions are made in sensitive areas of human life, such as whether a person will receive the requested credit, be classified as a repeat offender, be hired for a job, or be diagnosed with a particular disease.…”

“…Pre-processing mitigation strategies correspond to the stage of data collection in the ML pipeline and are used to ensure that the predefined protected feature does not impact the outcome negatively by modifying the feature space [20]. The biases inherent in the data itself are removed before model training to account for fair outcomes [22]. Bias mitigation at the stage of model development includes diverse approaches.…”

“…Bias mitigation at the stage of model development includes diverse approaches. In-processing mitigation strategies are modifications of model development, e.g., changing the loss function, adjusting the learning algorithm itself or compositional and adversarial approaches [22]. However, since these approaches are only suitable for certain optimization problems [20], in-processing mitigation strategies come with less generality.…”

“…Post-processing mitigation approaches occur at the stage of model evaluation. They are applied on potentially biased, trained classifiers to ensure fairness with respect to the sensitive traits of an individual [22]. Postprocessing is independent of the modeling process and, hence, applies to any black-box classifier.…”

“…Instead of re-sampling or re-labeling the data, representation learning is applied to mitigate biases arising at the beginning of the ML pipeline [22]. Optimized Pre-processing makes use of a probabilistic framework to generate a randomized mapping of the training and test sets that controls for discrimination.…”