Federated Machine Learning

Abstract

Today’s artificial intelligence still faces two major challenges. One is that, in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated-learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated-learning framework, which includes horizontal federated learning, vertical federated learning, and federated transfer learning. We provide definitions, architectures, and applications for the federated-learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allowing knowledge to be shared without compromising user privacy.

Estimating Forest Aboveground Carbon Storage in Hang-Jia-Hu Using Landsat TM/OLI Data and Random Forest Model

“…Moreover, studies have found that remote sensing data and its derived bands have good practicability for simulating forest AGC [35][36][37]; this is especially the case when combined with machine learning algorithms that allow for large scale automated analysis of high dimensional data from satellites [38]. The machine learning approach can derive rich information from remote sensing data as the input data, and continuously optimize the algorithm's performance via empirical learning to make the results more feasible and credible [39][40][41]. Remotely sensed datasets, combined with machine learning algorithms for intelligent estimation of forest AGC, support more efficient and precise observation and management of forest resources [42][43][44].…”

Differentially Private Publication of Vertically Partitioned Data

“…Although the local raw data is not exposed in FL setting, FL on its own still lacks theoretical privacy guarantees [33], and may leak sensitive information about the training data [36]. Therefore, the combination of FL and proper privacy-preserving mechanisms, such as DP [14], HE [30], MPC [17], etc., is a necessity to alleviate FL's privacy risks.…”

“…Vertical federated learning (VFL) [20] has been recognized as one of the effective solutions for encouraging enterprise-level data collaborations while respecting data privacy [36], required by the strict government regulations like Europe's General Data Privacy Regulations (GDPR [34]). Unlike horizontal federated learning (HFL) [25,6] setting in which the decentralized datasets share the same feature space but little intersection on the sample space, in VFL setting, the datasets of different organizations share the same or similar sample space but differ in feature space.…”

“…Since IRs might leak training data information [1,9], they are mostly protected by additively homomorphic encryption (HE) [20,29] and secure multi-party computation (MPC) [27] in existing VFL, thanks to the sufficient computation power and network bandwidth of enterprise-level participants. In addition, given by the similar settings and assumptions, most existing VFL algorithms follow traditional privacypreserving multi-party machine learning methods [36,4,18,3,8], by applying Taylor approximation to the loss functions, so that HE can be adopted to protect the calculation of polynomial tasks in VFL's joint training.…”

“…For example, a VFLbased logistic regression task takes few minutes to finish training if IR is not securely computed and transmitted, while an HE version of VFL-based logistic regression takes hours. In addition, as mentioned in [36], most existing VFL methods require a third-party collaborator to ensure data confidentiality during training process. Moreover, It is non-trivial task to approximate certain critical functions, e.g., loss function in machine learning models using only low-degree polynomials before HE, and naive approximation may lead to big errors and makes the solutions intractable [23].…”