Unsupervised Labelling of Stolen Handwritten Digit Embeddings with Density Matching

Abstract

Biometrics authentication is now widely deployed, and from that omnipresence comes the necessity to protect private data. Recent studies proved touchscreen handwritten digits to be a reliable biometrics. We set a threat model based on that biometrics: in the event of theft of unlabelled embeddings of handwritten digits, we propose a labelling method inspired by recent unsupervised translation algorithms. Provided a set of unlabelled embeddings known to have been produced by a Long Short Term Memory Recurrent Neural Network (LSTM RNN), we demonstrate that inferring their labels is possible. The proposed approach involves label-wise clustering of the embeddings and label identification of each group by matching their distribution to the label-relative classes of a comparison hand-crafted labeled set of embeddings. Cluster labelling is done through a two steps process including a genetic algorithm that finds the N-best matching hypotheses before a fine-tuning of those N-candidates. The proposed method was able to infer the correct labels on 100 randomised runs on different dataset splits.

Handwritten Digits Reconstruction from Unlabelled Embeddings

“…This LDA is then applied on the same data, to project the embeddings from U in a more discriminant space. This different dimension reduction technique focused on the digit clusters and digit classes will allow the [11] method to work identity and digit value trained encoders, as shown in section 5.1.…”

“…If solving this equation in D = 256 dimensions for 10 clusters gives an unique computational solution (which will be used for the rest of the paper), it is important to note that from a mathematical point of view, defining a rotation with more dimensions than pivot point gives infinite potential rotations. Details about the losses and the initialisation can be found in [11].…”

“…The method of [11] consists in inferring the most represented digit value in 10 K-means clusters of U. In this work, the embedding extractor was only trained to predict digit values.…”

“…First investigations in this area [11] show that inferring the most represented digits values in clusters of stolen embeddings is possible by statically aligning them to the attack set of labelled embeddings.…”

“…We propose to enhance the method of [11] to initialize training of an alignment flow of the distributions of the stolen and attack embeddings in full dimension. Using methods from [13][14][15], a LSTM based Mixture Density Recurrent Neural Network (MD-RNN) decoder is also trained to reconstruct the attack set of embeddings into their original drawings.…”