A Teradata content-based multimedia object manager for massively parallel architectures

O'Connell, William T.; Ieong, Ion Tim; Schrader, D.; Watson, C.; Au, Grace; Biliris, A.; Choo, Seungyeon; Colin, P.; Linderman, George C.; Panagos, Euthimios; Wang, J.; Walter, Thomas

doi:10.1145/233269.233321

Cited by 16 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Parallel databases are typically deployed in a sharednothing MPP architecture, such as Teradata [6], Aster Data [7] and Greenplum [8]. They are high performance computing platform.…”

Section: Related Workmentioning

confidence: 99%

A Simple and Efficient Parallel Approach to Large-Scale Railway Freight Data Analysis

Xie

Liu

2010

2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC)

View full text Add to dashboard Cite

Analysis of rail freight data is essential to make a better strategy for railway freight traffic organization. However, rail freight data are national-widely stored in data center of railway bureaus. Furthermore, the size of the data is very large and grows rapidly. Although analytical data processing has received much attention these years, little attention is paid to transparently processing data stored in remote data sources, particularly where the data are generated. In this paper, we present a simple and efficient parallel approach, RFDPS, to large-scale rail freight data processing. RFDPS provides a novel virtualization data model to integrate geographically distributed freight data. Besides, it can parallelize and optimize data processing automatically across multiple data center of railway bureaus. We run typical rail freight analysis applications in production environment to evaluate the efficiency of RFDPS. The results show that the performance of RFDPS is improved exponentially compared to the approach China's Ministry of Railways adopted before.

show abstract

“…Parallel databases are typically deployed in a sharednothing MPP architecture, such as Teradata [6], Aster Data [7] and Greenplum [8]. They are high performance computing platform.…”

Section: Related Workmentioning

confidence: 99%

A Simple and Efficient Parallel Approach to Large-Scale Railway Freight Data Analysis

Xie

Liu

2010

2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC)

View full text Add to dashboard Cite

show abstract

“…On the one hand, on some systems such as Teradata [3,22], system internal data in query processing is converted to strings to pass in UDFs, which is easy for the application developer to deal with but incurs significant overhead, particularly in per-tuple processing. As reported in [20,21], with SQL Server, no matter how simple a UDF is, it significantly underperforms compared to a system function or expression.…”

Section: The Problemsmentioning

confidence: 99%

“…For making UDFs practically tractable, on some systems such as SQL Server and Teradata [3,22], the input of a UDF is converted to a string from the values in the system internal formats for query processing. With such a mechanism, as reported in [20,21], no matter how simple a UDF is, it significantly underperforms compared with a system function or expression.…”

Section: Conclusion and Related Workmentioning

confidence: 99%

Efficiently support MapReduce-like computation models inside parallel DBMS

Chen

Therber²,

Hsu

et al. 2009

Proceedings of the 2009 International Database Engineering &Amp; Applications Symposium on - IDEAS '09

View full text Add to dashboard Cite

While parallel DBMSs do support large scale parallel query processing on partitioned data, the reach of more general applications relies on User Defined Functions (UDFs). However, the existent UDF technology is insufficient both conceptually and practically. A UDF is not a relation-in, relation-out operator, which restricts its ability to model complex applications defined on a set of tuples rather than on a single one, and to be composed with other relational operators in a query. Further, to interact with the query execution efficiently, a UDF must be coded with complex interactions with DBMS internal data structures and system calls which is often beyond the expertise of an analytics application developer.To solve these problems, we start with wrapping general applications with Relation Valued Functions (RVFs); then based on the notion of invocation patterns, we provide focused system support for efficiently integrating RVF execution into the query processing pipeline. We further distinguish the system responsibility and the user responsibility in RVF development, by separating an RVF into the RVF-Shell for dealing with system interaction, and the user-function for pure application logic, such that the RVF-Shell can be constructed in terms of high-level APIs. These mechanisms enable us to solve the essential problems in supporting MapReduce and other analytics computation models inside a parallel database engine: modeling complex applications, integrating them into query processing, and shielding analytics developers from DBMS internal details.Prototyped on a commercial and proprietary parallel database engine, our experience reveals the practical value of the proposed approaches.

show abstract

“…3.6.1.2 Term Context Instead of relying on term co-occurrence, some work uses the context (surrounding terms) of each term to construct the vectors that represent each term [Gauch and Wang, 1996]. The problem with the vectors given above is that they do not differentiate the senses of the words.…”

Section: 611mentioning

confidence: 99%