Smart App Attack: Hacking Deep Learning Models in Android Apps

Yujin, Huang,; Chen, Chunyang

doi:10.1109/tifs.2022.3172213

Cited by 13 publications

(10 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(3) finding a similar differentiable model from the Internet by comparing the features among models -the current mainstream on-device model format TFLite does not support some advanced functions like auto-differentiation to protect the deployed model. But attackers like the App Attack can find a differentiable model with similar structure and weights from the Internet [16].…”

Section: Reverse Engineering Tools For DL Modelsmentioning

confidence: 99%

“…These app package files can then be decompiled by off-the-shelf reverse-engineering tools (e.g., Apktool 5 and IDA Pro 6 ) to get the original DL model file. Although many on-device DL platforms do not support some advanced functions like backpropagation, attackers can assemble the model architecture and weights into a differentiable model format [17], or they can use software analysis methods to generate attacks for the target model [16,26]. In this paper, we will obfuscate the information of the model and underlying DL library to prevent the software analysis and model conversion tools from getting model's key information.…”

Section: Platform and Threat Modelmentioning

confidence: 99%

“…In addressing this RQ we aim to demonstrate the effectiveness of ModelObfuscator. Attackers can either extract ML model information from the model file [26] or analyze the structure of models to "outputs": [1], "op_type": "Conv2DOptions", "builtin_options": {stride_w: 1, • • • }, Obfuscated "inputs": [0], 510 "outputs": [1], "op_type": "Fvsfxf", "custom_options": {84,0,1,3,1, • • • }, collect similar models from the web for security exploitation [16,17]. Thus, we demonstrate the effectiveness of ModelObfuscator in obfuscating model files and structures.…”

Section: Rq1: Obfuscation Effectivenessmentioning

confidence: 99%

“…√ ': this model parsing method cannot extract information for all models.refer to a study by Li et al[26], in which the researchers attempt to extract a model's structure and weights in FlatBuffer.If ModelObfuscator is effective, the FlatBuffer extractor will be unable to parse the data of the obfuscated model and reverse it to the original one. (3) The third method involves finding a similar differentiable model from the Internet by comparing the features among models.Although the mainstream on-device model format TFLite does not support some advanced functions like auto-differentiation to protect the deployed model, attackers such as the App Attack can find a differentiable model with a similar structure and weights from the Internet[16]. For the App Attacking method, if it can correctly identify the obfuscated model that has the same model structure as the original one on TensorFlow Hub, we consider it a successful extraction of the model information.…”

mentioning

confidence: 99%

See 3 more Smart Citations

ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-Based Systems

Zhou

Gao

et al. 2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Self Cite

View full text Add to dashboard Cite

More and more edge devices and mobile apps are leveraging deep learning (DL) capabilities. Deploying such models on devices -referred to as on-device models -rather than as remote cloud-hosted services, has gained popularity because it avoids transmitting user's data off of the device and achieves high response time. However, on-device models can be easily attacked, as they can be accessed by unpacking corresponding apps and the model is fully exposed to attackers. Recent studies show that attackers can easily generate white-box-like attacks for an on-device model or even inverse its training data. To protect on-device models from white-box attacks, we propose a novel technique called model obfuscation. Specifically, model obfuscation hides and obfuscates the key information -structure, parameters and attributes -of models by renaming, parameter encapsulation, neural structure obfuscation obfuscation, shortcut injection, and extra layer injection. We have developed a prototype tool ModelObfuscator to automatically obfuscate on-device TFLite models. Our experiments show that this proposed approach can dramatically improve model security by significantly increasing the difficulty of parsing models' inner information, without increasing the latency of DL models. Our proposed on-device model obfuscation has the potential to be a fundamental technique for ondevice model deployment. Our prototype tool is publicly available at https://github.com/zhoumingyi/ModelObfuscator.

show abstract

Section: Reverse Engineering Tools For DL Modelsmentioning

confidence: 99%

Section: Platform and Threat Modelmentioning

confidence: 99%

Section: Rq1: Obfuscation Effectivenessmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

ModelObfuscator: Obfuscating Model Information to Protect Deployed ML-Based Systems

Zhou

Gao

et al. 2023

Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis

Self Cite

View full text Add to dashboard Cite

show abstract

“…In fact, many vulnerabilities of on-device models have already been discovered by our fellow researchers in recent years. For example, Huang et al [15,16] propose to achieve the purpose by parsing features of the on-device model to find a surrogate model from the web, which could then be used to launch transferable attacks on mobile models. Cao et al [4] also use surrogate models for attacking mobile models under the black-box setting, albeit by obtaining information from mobile models via querying their outputs, and then training a surrogate model using such information.…”

Section: Introductionmentioning

confidence: 99%

Investigating White-Box Attacks for On-Device Models

Zhou,

Gao,

et al. 2024

Proceedings of the IEEE/ACM 46th International Conference on Software Engineering

View full text Add to dashboard Cite

Numerous mobile apps have leveraged deep learning capabilities. However, on-device models are vulnerable to attacks as they can be easily extracted from their corresponding mobile apps. Although the structure and parameters information of these models can be accessed, existing on-device attacking approaches only generate black-box attacks (i.e., indirect white-box attacks), which are less effective and efficient than white-box strategies. This is because mobile deep learning (DL) frameworks like TensorFlow Lite (TFLite) do not support gradient computing (referred to as non-debuggable models), which is necessary for white-box attacking algorithms. Thus, we argue that existing findings may underestimate the harmfulness of on-device attacks. To validate this, we systematically analyze the difficulties of transforming the on-device model to its debuggable version and propose a Reverse Engineering framework for On-device Models (REOM), which automatically reverses the compiled on-device TFLite model to its debuggable version, enabling attackers to launch white-box attacks. Our empirical results show that our approach is effective in achieving automated transformation (i.e., 92.6%) among 244 TFLite models. Compared with previous attacks using surrogate models, REOM enables attackers to achieve higher attack success rates (10.23%→89.03%) with a hundred times smaller attack perturbations (1.0→0.01). Our findings emphasize the need for developers to carefully consider their model deployment strategies, and use white-box methods to evaluate the vulnerability of on-device models. Our artifacts 1 are available.

show abstract