Semantic anomaly detection in online data sources

Raz, Orna; Koopman, Philip; Shaw, Mary

doi:10.1145/581376.581378

Cited by 43 publications

(30 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, Cues infers constraints over web service data [14], and Forms/3 infers numeric constraints over spreadsheet cells [2]. From a conceptual standpoint, Toped generalizes these number-oriented systems to include string-like data.…”

Section: Related Workmentioning

confidence: 99%

Fast, Accurate Creation of Data Validation Formats by End-User Developers

Scaffidi

Myers

Shaw

2009

End-User Development

Self Cite

View full text Add to dashboard Cite

Abstract. Inputs to web forms often contain typos or other errors. However, existing web form design tools require end-user developers to write regular expressions ("regexps") or even scripts to validate inputs, which is slow and errorprone because of the poor match between common data types and the regexp notation. We present a new technique enabling end-user developers to describe data as a series of constrained parts, and we have incorporated our technique into a prototype tool. Using this tool, end-user developers can create validation code more quickly and accurately than with existing techniques, finding 90% of invalid inputs in a lab study. This study and our evaluation of the technique's generality have motivated several tool improvements, which we have implemented and now evaluate using the Cognitive Dimensions framework.

show abstract

Section: Related Workmentioning

confidence: 99%

Fast, Accurate Creation of Data Validation Formats by End-User Developers

Scaffidi

Myers

Shaw

2009

End-User Development

Self Cite

View full text Add to dashboard Cite

show abstract

“…This research draws from results of software engineering research into how to support phases of the software lifecycle beyond the coding stage. Combining this research with visual expressions, machine learning techniques, and emerging HCI research, researchers are now developing new software engineering methodologies to help end-user programmers reduce defects by supporting them beyond the coding stage, such as with incremental visual testing [Rothermel et al 1998], semi-automatic defect detection [Bottoni et al 1997;Miller and Myers 2001;Raz et al 2002], and tightly integrated assertion mechanisms [Burnett et al 2003]. …”

Section: Software Engineering Trends In Modern Visual Programming Resmentioning

confidence: 99%

The impact of software engineering research on modern programming languages

Ryder

Soffa

Burnett

2005

ACM Trans. Softw. Eng. Methodol.

View full text Add to dashboard Cite

Software engineering research and programming language design have enjoyed a symbiotic relationship, with traceable impacts since the 1970s, when these areas were first distinguished from one another. This report documents this relationship by focusing on several major features of current programming languages: data and procedural abstraction, types, concurrency, exceptions, and visual programming mechanisms. The influences are determined by tracing references in publications in both fields, obtaining oral histories from language designers delineating influences on them, and tracking cotemporal research trends and ideas as demonstrated by workshop topics, special issue publications, and invited talks in the two fields. In some cases there is conclusive This article has been developed under the auspices of the Impact Project. The aim of the project is to provide a scholarly study of the impact that software engineering research-both academic and industrial-has had upon practice. The principal output of the project is a series of individual papers covering the impact upon practice of research in several selected major areas of software engineering. Each of these papers is being published in ACM TOSEM. Additional information about the project can be found at http://www.acm.org/sigsoft/impact/. This article is based upon work supported by the U.S. National Science Foundation (NSF) under award number CCF-0137766, the Association of Computing Machinery Special Interest Group on Software Engineering (ACM SIGSOFT), the Institution of Electrical Engineers (IEE), and the Japan Electronics and Information Technology Association (JEITA). Any opinions, findings and conclusions or recommendations, expressed in this publication are those of the authors and do not necessarily reflect, of the NSF, ACM SIGSOFT, the IEE, or JEITA. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or direct commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 1515 Broadway, New York, NY 10036 USA, fax: +1 (212) 869-0481, or permissions@acm.org. data supporting influence. In other cases, there are circumstantial arguments (i.e., cotemporal ideas) that indicate influence. Using this approach, this study provides evidence of the impact of software engineering research on modern programming language design and documents the close relationship between these two fields.

show abstract

“…In [23] we proposed a method for inferring adaptive invariants about the normal behavior of dynamic data feeds, using and adapting existing techniques for the inference. Adaptive invariants are not only static invariants but also relations and values that are expected to change as the behavior of the data changes.…”

Section: Detectionmentioning

confidence: 99%

“…We re-examine one of the stock quote data feed experiments we ran in [23] to show how the template mechanism creates and updates this classification. The templates seem to stabilize rather quickly (around 3 iterations).…”

Section: Examplementioning

confidence: 99%

Enabling automatic adaptation in systems with under-specified elements

Raz

Koopman

Shaw

2002

Proceedings of the First Workshop on Self-Healing Systems

Self Cite

View full text Add to dashboard Cite

Software that people use for everyday purposes is usually not mission critical-some failures can be tolerated. However, this software should be dependable enough for its intended use, even when users change expectations. Software systems that could adapt to accommodate both failures and changing user expectations could significantly improve the dependability of such everyday software. Many adaptation techniques require specifications of proper behavior (for detecting improper behavior) and problem severity, alternatives and their selection (for mitigation and for repair).However, the specifications of everyday software are usually incomplete and imprecise. This makes it difficult to determine the dependability of the software and even more difficult to adapt.We address the problem of detecting anomalies-deviations from expected behavior-when specifications of expected behavior are missing. Setting up anomaly detection depends on human participation, yielding predicates that can serve as proxies for missing specifications.We propose a template mechanism to lower the demands on human attention when setting up detection. We show how this mechanism may be used in our framework for enhancing dynamic data feeds with automatic adaptation. We discuss how the same mechanism may be used in repair. Our emphasis is on detecting semantic anomalies: cases in which the data feed is responsive and delivers well-formed results, but these results are unreasonable.

show abstract

Semantic anomaly detection in online data sources

Cited by 43 publications

References 0 publications

Fast, Accurate Creation of Data Validation Formats by End-User Developers

Fast, Accurate Creation of Data Validation Formats by End-User Developers

The impact of software engineering research on modern programming languages

Enabling automatic adaptation in systems with under-specified elements

Contact Info

Product

Resources

About