Enhancing Sensitivity Classification with Semantic Features Using Word Embeddings

McDonald, Graham; Macdonald, Craig; Ounis, Iadh

doi:10.1007/978-3-319-56608-5_35

Cited by 26 publications

(32 citation statements)

References 19 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Feature engineering for sensitivity classification was subsequently investigated further by McDonald et al [5]. In that work, the authors constructed document representations using word embeddings to capture semantic relations in the documents, such as who said what about whom.…”

Section: Related Workmentioning

confidence: 99%

“…However, in the era of born-digital documents such as e-mail, this purely manual review is not feasible [1], for example due to the volume of digital documents that are to be reviewed. Recently, automatic sensitivity classification algorithms have been shown to have potential for effectively identifying sensitive information in documents [2][3][4][5]. However, the potential consequences from the inadvertent release of sensitive information can be severe, for example if the identity of an informant is made public it can put the informant and their family at risk.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Active Learning Strategies for Technology Assisted Sensitivity Review

McDonald

Macdonald

Ounis

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Government documents must be reviewed to identify and protect any sensitive information, such as personal information, before the documents can be released to the public. However, in the era of digital government documents, such as e-mail, traditional sensitivity review procedures are no longer practical, for example due to the volume of documents to be reviewed. Therefore, there is a need for new technology assisted review protocols to integrate automatic sensitivity classification into the sensitivity review process. Moreover, to effectively assist sensitivity review, such assistive technologies must incorporate reviewer feedback to enable sensitivity classifiers to quickly learn and adapt to the sensitivities within a collection, when the types of sensitivity are not known a priori. In this work, we present a thorough evaluation of active learning strategies for sensitivity review. Moreover, we present an active learning strategy that integrates reviewer feedback, from sensitive text annotations, to identify features of sensitivity that enable us to learn an effective sensitivity classifier (0.7 Balanced Accuracy) using significantly less reviewer effort, according to the sign test (p < 0.01). Moreover, this approach results in a 51% reduction in the number of documents required to be reviewed to achieve the same level of classification accuracy, compared to when the approach is deployed without annotation features.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Active Learning Strategies for Technology Assisted Sensitivity Review

McDonald

Macdonald

Ounis

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Assisting the sensitivity review of digital government documents has received some attention in the literature in recent years [4][5][6][7][8][9]. Most of that work has focused on developing classification algorithms for identifying sensitivity, either at the document level [5,7] or sensitive text within documents [9].…”

Section: Related Workmentioning

confidence: 99%

“…Most of that work has focused on developing classification algorithms for identifying sensitivity, either at the document level [5,7] or sensitive text within documents [9]. Berardi et al [8] investigated improving the cost-effectiveness of sensitivity reviewers by deploying a utility-theoretic [10] semi-automatic text classification approach to identify a ranking strategy that can maximise the overall classification effectiveness when a reviewer corrects a portion of misclassified documents, i.e.…”

Section: Related Workmentioning

confidence: 99%

Towards Maximising Openness in Digital Sensitivity Review Using Reviewing Time Predictions

McDonald

Macdonald

Ounis

2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. The adoption of born-digital documents, such as email, by governments, such as in the UK and USA, has resulted in a large backlog of born-digital documents that must be sensitivity reviewed before they can be opened to the public, to ensure that no sensitive information is released, e.g. personal or confidential information. However, it is not practical to review all of the backlog with the available reviewing resources and, therefore, there is a need for automatic techniques to increase the number of documents that can be opened within a fixed reviewing time budget. In this paper, we conduct a user study and use the log data to build models to predict reviewing times for an average sensitivity reviewer. Moreover, we show that using our reviewing time predictions to select the order that documents are reviewed can markedly increase the ratio of reviewed documents that are released to the public, e.g. +30% for collections with high levels of sensitivity, compared to reviewing by shortest document first. This, in turn, increases the total number of documents that are opened to the public within a fixed reviewing time budget, e.g. an extra 200 documents in 100 hours reviewing.

show abstract

“…During the poster session, Graham presented his evaluation of the effectiveness of semantic word embedding features, along with term and grammatical features, for sensitivity classification. On a test collection of government documents containing real sensitivities, he showed that extending text classification with semantic features and additional term n-grams results in significant improvements in classification effectiveness, correctly classifying 9.99% more sensitive documents compared to the text classification baseline [20][21][22].…”

Section: An Initial Investigation Into Fixed and Adaptive Stopping Stmentioning

confidence: 99%

Report on the Information Retrieval Festival (IRFest2017)

et al. 2017

Self Cite

View full text Add to dashboard Cite

The Information Retrieval Festival took place in April 2017 in Glasgow. The focus of the workshop was to bring together IR researchers from the various Scottish universities and beyond in order to facilitate more awareness, increased interaction and reflection on the status of the field and its future. The program included an industry session, research talks, demos and posters as well as two keynotes. The first keynote was delivered by Prof. Jaana Kekalenien, who provided a historical, critical reflection of realism in Interactive Information Retrieval Experimentation, while the second keynote was delivered by Prof. Maarten de Rijke, who argued for more Artificial Intelligence usage in IR solutions and deployments. The workshop was followed by a "Tour de Scotland" where delegates were taken from Glasgow to Aberdeen for the European Conference in Information Retrieval (ECIR 2017). OverviewOn the 7th of April, 2017, the 2nd SICSA Scottish Information Retrieval (IR) Workshop was held at the University of Glasgow. The workshop was co-organised by the University of Glasgow and the University of Strathclyde, and was funded by the Scottish Informatics and Computing Science Alliance (SISCA), Glasgow School of Computing and supported by ACM SIGIR.The aim of the workshop was to increase awareness and interaction between IR-related researchers working in, across, and with Scottish and UK universities.The workshop had a total of 75 attendees from across the Scottish universities and beyond. The scientific program included keynotes, industry talks, lightening academic talks and a poster/demo session as follows: 2 academic keynotes, 4 industrial talks and 12 invited academic talks from 7 different research institutions

show abstract

Enhancing Sensitivity Classification with Semantic Features Using Word Embeddings

Cited by 26 publications

References 19 publications

Active Learning Strategies for Technology Assisted Sensitivity Review

Active Learning Strategies for Technology Assisted Sensitivity Review

Towards Maximising Openness in Digital Sensitivity Review Using Reviewing Time Predictions

Report on the Information Retrieval Festival (IRFest2017)

Contact Info

Product

Resources

About