Adaptive learning of aggregate analytics under dynamic workloads

Savva, Fotis; Anagnostopoulos, Christos; Triantafillou, Peter

doi:10.1016/j.future.2020.03.063

Cited by 9 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, they propose a multiple Q-tables scheme as knowledge base of the QC in the RL case and a technique for deriving the level of compactness of the created clusters in the clustering scheme, to deliver the best possible QP for each assignment. In [3] the authors introduce an adaptive, reciprocity-based Machine Learning mechanism, to estimate the answers of a variety of aggregate queries (AQs) avoiding the big data back-end. The mechanism learns from past analytical-query patterns while they develop solutions to correspond in the changes in queries' analytics and analysts' interests.…”

Section: Related Workmentioning

confidence: 99%

Query Driven Data Subspace Mapping

Fountas

Papathanasaki

Kolomvatsos

et al. 2022

IFIP Advances in Information and Communication Technology

Self Cite

View full text Add to dashboard Cite

The increased use of multiple types of computer systems and smart devices in several areas, has created massive amounts of data. Concurrently, the need for a subset of these data by numerous applications and users for task execution and knowledge extraction, has resulted in the injection of a massive number of queries per second into distributed database servers. As a result of this phenomena, a major process is the efficient response of these queries both in terms of time and the detection of acceptable data, while rejecting undesired data points. In this paper, we present a hierarchical query-driven clustering approach, for performing efficient data mapping in remote databases for future incoming queries. We distinguish ourselves from current methods, by combining the technique of Query-Based Learning (QBL) with a hierarchical clustering of multiple forms of clustering in the same model. The suggested model's performance is assessed, using a number of experimental scenarios as well as numerical data.

show abstract

Section: Related Workmentioning

confidence: 99%

Query Driven Data Subspace Mapping

Fountas

Papathanasaki

Kolomvatsos

et al. 2022

IFIP Advances in Information and Communication Technology

Self Cite

View full text Add to dashboard Cite

show abstract

“…A new distribution might signify new user interests and application requirements. Studying the full effects of this has been studied in the context of Approximate Query Processing [44][45][46], but not in the context of explanation functions, which remains part of our future work.…”

Section: Off-line Adjustmentmentioning

confidence: 99%

Large-scale Data Exploration Using Explanatory Regression Functions

Savva

Anagnostopoulos

Triantafillou

et al. 2020

ACM Trans. Knowl. Discov. Data

Self Cite

View full text Add to dashboard Cite

Analysts wishing to explore multivariate data spaces, typically issue queries involving selection operators, i.e., range or equality predicates, which define data subspaces of potential interest. Then, they use aggregation functions, the results of which determine a subspace's interestingness for further exploration and deeper analysis. However, Aggregate Query (AQ) results are scalars and convey limited information and explainability about the queried subspaces for enhanced exploratory analysis. Analysts have no way of identifying how these results are derived or how they change w.r.t query (input) parameter values. We address this shortcoming by aiding analysts to explore and understand data subspaces by contributing a novel explanation mechanism based on Machine Learning. We explain AQ results using functions obtained by a threefold joint optimization problem which assume the form of explainable piecewise-linear regression functions. A key feature of the proposed solution is that the explanation functions are estimated using past executed queries. These queries provide a coarse grained overview of the underlying aggregate function (generating the AQ results) to be learned. Explanations for future, previously unseen AQs can be computed without accessing the underlying data and can be used to further explore the queried data subspaces, without issuing more queries to the backend analytics engine. We evaluate the explanation accuracy and efficiency through theoretically grounded metrics over real-world and synthetic datasets and query workloads. CCS Concepts: • Mathematics of computing → Exploratory data analysis; • Information systems → Data analytics; • Computing methodologies → Supervised learning by regression.

show abstract

“…Proof 1: Proof is omitted and can be found at [19] The convergence of the representatives is checked by the subsequent adjustments in positions that w K+1 makes. If that change is lower than a threshold c then convergence has been achieved.…”

Section: A Model Adaptation and Reciprocitymentioning

confidence: 99%

Aggregate Query Prediction under Dynamic Workloads

Savva

Anagnostopoulos

Triantafillou

2019

2019 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

Large organizations have seamlessly incorporated data-driven decision making in their operations. However, as data volumes increase, expensive big data infrastructures are called to rescue. In this setting, analytics tasks become very costly in terms of query response time, resource consumption, and money in cloud deployments, especially when base data are stored across geographically distributed data centers. Therefore, we introduce an adaptive Machine Learning mechanism which is lightweight , stored client-side, can estimate the answers of a variety of aggregate queries and can avoid the big data backend. The estimations are performed in milliseconds and are inexepensive as the mechanism learns from past analytical-query patterns. However, as analytic queries are ad-hoc and analysts' interests change over time we develop solutions that can swiftly and accurately detect such changes and adapt to new query patterns. The capabilities of our approach are demonstrated using extensive evaluation with real and synthetic datasets.

show abstract

Adaptive learning of aggregate analytics under dynamic workloads

Cited by 9 publications

References 18 publications

Query Driven Data Subspace Mapping

Query Driven Data Subspace Mapping

Large-scale Data Exploration Using Explanatory Regression Functions

Aggregate Query Prediction under Dynamic Workloads

Contact Info

Product

Resources

About