Secure tumor classification by shallow neural network using homomorphic encryption

Abstract

Background Disclosure of patients’ genetic information in the process of applying machine learning techniques for tumor classification hinders the privacy of personal information. Homomorphic Encryption (HE), which supports operations between encrypted data, can be used as one of the tools to perform such computation without information leakage, but it brings great challenges for directly applying general machine learning algorithms due to the limitations of operations supported by HE. In particular, non-polynomial activation functions, including softmax functions, are difficult to implement with HE and require a suitable approximation method to minimize the loss of accuracy. In the secure genome analysis competition called iDASH 2020, it is presented as a competition task that a multi-label tumor classification method that predicts the class of samples based on genetic information using HE. Methods We develop a secure multi-label tumor classification method using HE to ensure privacy during all the computations of the model inference process. Our solution is based on a 1-layer neural network with the softmax activation function model and uses the approximate HE scheme. We present an approximation method that enables softmax activation in the model using HE and a technique for efficiently encoding data to reduce computational costs. In addition, we propose a HE-friendly data filtering method to reduce the size of large-scale genetic data. Results We aim to analyze the dataset from The Cancer Genome Atlas (TCGA) dataset, which consists of 3,622 samples from 11 types of cancers, genetic features from 25,128 genes. Our preprocessing method reduces the number of genes to 4,096 or less and achieves a microAUC value of 0.9882 (85% accuracy) with a 1-layer shallow neural network. Using our model, we successfully compute the tumor classification inference steps on the encrypted test data in 3.75 minutes. As a result of exceptionally high microAUC values, our solution was awarded co-first place in iDASH 2020 Track 1: “Secure multi-label Tumor classification using Homomorphic Encryption”. Conclusions Our solution is the first result of implementing a neural network model with softmax activation using HE. Also, HE optimization methods presented in this work enable machine learning implementation using HE or other challenging HE applications.

Discovering Millennials’ Migration Clusters in Seoul, South Korea: A Local Spatial Network Autocorrelation Approach

“…It shows how the minimum and maximum values of input of softmax vary as the training proceeds. The value increases in the order of two, which cannot be handled by previous approximation methods (Lee et al, 2022b;Hong et al, 2022;Jin et al, 2020). This shows that, with the previous approximation methods, it is hard to train a model as much as we want.…”

“…There are several works on the approximation of softmax function with polynomials (Lee et al, 2022b;Hong et al, 2022;Jin et al, 2020). However, all of these methods have low precision (See the Table 3 in the Appendix) and permit only a small domain of approximation, which is not desirable for training with many epochs.…”

“…However, all of these methods have low precision (See the Table 3 in the Appendix) and permit only a small domain of approximation, which is not desirable for training with many epochs. We remark that both (Lee et al, 2022b) and (Hong et al, 2022) have been used for inference rather than training.…”

“…This also fits into our case for computing XW ⊺ , because the number of rows of W equals the number of classes for the dataset, which is often small compared to s 0 or s 1 . In addition, complexification reduces the complexity by half (a similar idea was used in (Hong et al, 2022)). Finally, we can extend the algorithm to compute tAB ⊺ for t ∈ R without additional multiplicative depth consumption by replacing the diagonal mask…”

“…The authors of (Lee et al, 2022b) used the Gumbel softmax function instead of the original softmax function to make the input be in the approximation region. In (Hong et al, 2022), the authors used the approximation AE r,L (x) = ((2 r + x)/L) 2 r for the scaled exponential function and combined it with Goldschmidt's algorithm to obtain an approximation of softmax. However, one must carefully select the scaling factor L, which is generally difficult.…”