Towards Maximising Openness in Digital Sensitivity Review Using Reviewing Time Predictions

Abstract

Abstract. The adoption of born-digital documents, such as email, by governments, such as in the UK and USA, has resulted in a large backlog of born-digital documents that must be sensitivity reviewed before they can be opened to the public, to ensure that no sensitive information is released, e.g. personal or confidential information. However, it is not practical to review all of the backlog with the available reviewing resources and, therefore, there is a need for automatic techniques to increase the number of documents that can be opened within a fixed reviewing time budget. In this paper, we conduct a user study and use the log data to build models to predict reviewing times for an average sensitivity reviewer. Moreover, we show that using our reviewing time predictions to select the order that documents are reviewed can markedly increase the ratio of reviewed documents that are released to the public, e.g. +30% for collections with high levels of sensitivity, compared to reviewing by shortest document first. This, in turn, increases the total number of documents that are opened to the public within a fixed reviewing time budget, e.g. an extra 200 documents in 100 hours reviewing.

The Role of Latent Semantic Categories and Clustering in Enhancing the Efficiency of Human Sensitivity Review

“…We note that the output of our proposed prioritisation strategies that rank clusters by their sensitivity classification probabilities can be dependent on the effectiveness of the deployed sensitivity classifier. Moreover, reviewing shorter documents first may not always lead to a faster review of documents, as is described by McDonald et al [14]. In this study, we control these variables as constant across all the aforementioned review prioritisation strategies to isolate the effectiveness of our proposed Cluster+Metadata approach.…”

“…Differently from the work of Hutchison, in this work, we study document clustering for sensitivity review in an interactive setting to evaluate the impact of reviewing documents in semantic clusters, on the efficiency, accuracy, and openness of sensitivity review. McDonald et al [14] proposed a method of predicting the amount of time that a reviewer would need to review a collection of documents and show that their proposed method can notably improve openness by prioritising the documents that are predicted to take less time to review. In contrast, in this work, we present a method of prioritising groups of documents for review to maximise openness.…”

Sensitivity Review of Large Collections by Identifying and Prioritising Coherent Documents Groups

“…For example, the details of an employee's salary are more likely to be sensitive in documents about business discussions than mentions of salaries in documents about political discussions, since politicians' salaries are usually in the public domain. Prioritising particular groups of related documents for review can also help to increase the number of documents that can be opened to the public when there are limited reviewing resources [9] (i.e., openness [6]). However, in large unstructured document collections, it is not practical for reviewers to manually identify such groups of related documents.…”

How the Accuracy and Confidence of Sensitivity Classification Affects Digital Sensitivity Review

“…For example, by prioritising not-sensitive documents to increase the number of non-sensitive documents that can be released to the public with limited reviewing resources [McDonald et al 2018]. In this work, we focus on evaluating if providing the reviewers with sensitivity classification predictions can reduce the time that it takes for a reviewer to review a collection of documents, while maintaining (or increasing) the reviewing accuracy.…”