DataPrep.EDA: Task-Centric Exploratory Data Analysis for Statistical Modeling in Python

Peng, Jinglin; Wu, Weiyuan; Lockhart, Brandon; Bian, Song; Yan, Jing Nathan; Xu, Linghao; Chi, Zhixuan; Rzeszotarski, Jeffrey M.; Wang, Jiannan

doi:10.1145/3448016.3457330

Cited by 21 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Upon examination of the dataset, insights provided by a Python Data Prep library [64] shows three of the numerical features (namely, BTC, USD, and Netflow Bytes) exhibiting significant skewness in their distributions (Figure 5). A sequence of mathematical adjustments/transformations [65] was applied to these characteristics to address their skewed distributions.…”

Section: A Data Pre-processingmentioning

confidence: 99%

RFSA: A Ransomware Feature Selection Algorithm for Multivariate Analysis of Malware Behavior in Cryptocurrency

Nkongolo Wa Nkongolo

2024

IJCDS

View full text Add to dashboard Cite

This research introduces innovative features tailored to capture distinctive characteristics of ransomware activity within the cryptocurrency ecosystem. The study employs a multifaceted analysis to delve into ransomware-related data encompassing transaction metadata, ransom analysis, behavioral patterns, and financial aspects. A feature selection algorithm is explored to discern ransomware transactions in Bitcoin (BTC) and the United States Dollar (USD) using the UGRansome dataset. This comprehensive dataset of ransomware-related transactions facilitates the proposal of novel features designed to capture the unique traits of ransomware activity. The correlation matrix and temporal analysis of these features contribute to a nuanced understanding of the dynamic nature of ransomware threats. The research presents the Ransomware Feature Selection Algorithm (RFSA) based on Gini Impurity and Mutual Information (MI) to effectively select crucial ransomware features. Evaluation metrics such as precision, recall, accuracy, and F1 score highlight the effectiveness of the RFSA. The analysis reveals that approximately 68% of ransomware incidents involve BTC transactions ranging from 1.46 to 2.56, with an average of 2.01 BTC transactions per attack. Moreover, ransomware causes financial damages ranging from 4.38 to 172.36 USD, with an average damage of 88.37 USD. The RFSA identifies 17 ransomware types and their associated malware to shed light on their characteristics. The study investigates the pricing of ransomware and reveals that TowerWeb is associated with the highest fee, amounting to 135.26 BTC, while CryptoLocker has the lowest fee, recorded at 10.51 BTC. Additionally, the impact of ransomware duration on financial gains and network flow is investigated, disclosing a correlation between extended duration and higher financial gains. The research achieves outstanding performance metrics, including an MI score of 95%, accuracy of 93%, recall of 92%, and precision of 89%. These results showcase the superiority of the proposed approach over existing studies, emphasizing the dynamic and adaptable nature of ransomware demands. The findings suggest that there is no fixed amount for specific cyberattacks. This underscores the importance of adapting to the evolving landscape of ransomware threats.

show abstract

Section: A Data Pre-processingmentioning

confidence: 99%

RFSA: A Ransomware Feature Selection Algorithm for Multivariate Analysis of Malware Behavior in Cryptocurrency

Nkongolo Wa Nkongolo

2024

IJCDS

View full text Add to dashboard Cite

show abstract

“…Preprocessing data: Tahap ini merupakan tahap persiapan data sebelum dilakukan analisis dengan heatmap korelasi dan scatterplot. Preprocessing data meliputi pembersihan data, pengubahan data menjadi bentuk yang sesuai dengan kebutuhan analisis, dan penghapusan data yang tidak valid [7]. 4.…”

Section: Metode Penelitianunclassified

Analisis Heatmap Korelasi dan Scatterplot untuk Mengidentifikasi Faktor-Faktor yang Mempengaruhi Pelabelan AC efisiensi Energi

2023

j.n.a.material.manufaktur.dan.n.a

View full text Add to dashboard Cite

This research aims to evaluate the factors that affect AC energy efficiency using statistical analysis methods. The data used is energy-efficient AC labeling data obtained from the Directorate General of New, Renewable, and Energy Conservation Energy (EBETKE) database. Violin plots are used to see the distribution of the data, a correlation heatmap is used to display the level of correlation between variables, and scatterplots and R-squared values are used to visualize linear relationships and measure the strength of the relationship. The results of the study show that efficiency has a very strong positive correlation with rating, with a correlation coefficient of 0.75, while it has a weak negative correlation with other variables. The R-squared value obtained for the linear relationship between efficiency and rating is 0.56, which indicates that 56% of the variation in efficiency can be explained by the variation in rating. This result shows that rating is a very influential factor on AC energy efficiency.

show abstract

“…Based on feedback from our domain expert interviewees, we model three of these dimensions (completeness, free-of-error, and objectivity) to guide users about potential data quality concerns. While data cleaning (e.g., imputing missing values) [18,20,47,60,72,73,76,88,94,104,125] is deferred to future work, DataPilot currently provides novel GUI interaction affordances to support subset selection.…”

Section: Datamentioning

confidence: 99%

DataPilot: Utilizing Quality and Usage Information for Subset Selection during Visual Data Preparation

Narechania

Sinha³

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Selecting relevant data subsets from large, unfamiliar datasets can be difficult. We address this challenge by modeling and visualizing two kinds of auxiliary information: (1) quality -the validity and appropriateness of data required to perform certain analytical tasks; and (2) usage -the historical utilization characteristics of data across multiple users. Through a design study with 14 data workers, we integrate this information into a visual data preparation and analysis tool, DataPilot. DataPilot presents visual cues about "the good, the bad, and the ugly" aspects of data and provides graphical user interface controls as interaction affordances, guiding users to perform subset selection. Through a study with 36 participants, we investigate how DataPilot helps users navigate a large, unfamiliar tabular dataset, prepare a relevant subset, and build a visualization dashboard. We find that users selected smaller, effective subsets with higher quality and usage, and with greater success and confidence. CCS CONCEPTS• Human-centered computing → Empirical studies in visualization; Visualization systems and tools.

show abstract

DataPrep.EDA: Task-Centric Exploratory Data Analysis for Statistical Modeling in Python

Cited by 21 publications

References 23 publications

RFSA: A Ransomware Feature Selection Algorithm for Multivariate Analysis of Malware Behavior in Cryptocurrency

RFSA: A Ransomware Feature Selection Algorithm for Multivariate Analysis of Malware Behavior in Cryptocurrency

Analisis Heatmap Korelasi dan Scatterplot untuk Mengidentifikasi Faktor-Faktor yang Mempengaruhi Pelabelan AC efisiensi Energi

DataPilot: Utilizing Quality and Usage Information for Subset Selection during Visual Data Preparation

Contact Info

Product

Resources

About