New algorithms assessing short summaries in expository texts using latent semantic analysis

Olmos, Ricardo; Gascón, José Antonio León; Jorge-Botana, Guillermo; Escudero, Isabel

doi:10.3758/brm.41.3.944

Cited by 24 publications

(22 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evidence that search patterns are not exclusively guided by visual properties of a scene has been forthcoming from experiments showing both that visual search can be made less efficient by semantically related distractors (e.g., Belke et al, 2008) and that semantic relationships between objects in a real-world scene (measured, for example, by distributional semantic models such as Latent Semantic Analysis; Landauer, Foltz, & Laham, 1998) are better predictors of eye-movement patterns than relationships based on visual similarity alone, though the latter also have some predictive power (Hwang et al, 2011). In a related vein, eye-movement research by Henderson et al (2007) showed that the fixated regions of a complex scene tended to be those with lower intensity and high local contrast; however, these were also regions that were independently judged as having high semantic content.…”

Section: Conclusion: Beyond Static Artificial Scenes?mentioning

confidence: 99%

Reference Production as Search: The Impact of Domain Size on the Production of Distinguishing Descriptions

Gatt

Krahmer

Deemter³

et al. 2016

Cognitive Science

View full text Add to dashboard Cite

When producing a description of a target referent in a visual context, speakers need to choose a set of properties that distinguish it from its distractors. Computational models of language production/generation usually model this as a search process and predict that the time taken will increase both with the number of distractors in a scene and with the number of properties required to distinguish the target. These predictions are reminiscent of classic findings in visual search; however, unlike models of reference production, visual search models also predict that search can become very efficient under certain conditions, something that reference production models do not consider. This paper investigates the predictions of these models empirically. In two experiments, we show that the time taken to plan a referring expression-as reflected by speech onset latencies-is influenced by distractor set size and by the number of properties required, but this crucially depends on the discriminability of the properties under consideration. We discuss the implications for current models of reference production and recent work on the role of salience in visual search.

show abstract

Section: Conclusion: Beyond Static Artificial Scenes?mentioning

confidence: 99%

Reference Production as Search: The Impact of Domain Size on the Production of Distinguishing Descriptions

Gatt

Krahmer

Deemter³

et al. 2016

Cognitive Science

View full text Add to dashboard Cite

show abstract

“…This fault seems to be corrected with the use of Euclidean distances as a measure of similarity. We have called this phenomenon, previously observed in other studies (Olmos et al, 2009), the ''distance corrector effect''. Since Non-expert responses do not meet minimum length and content requirements, the ''distance corrector effect'' is critical in evaluating this group.…”

Section: Discussionmentioning

confidence: 79%

“…One of the hypotheses is that the Euclidean distance (measuring similarity) corrects the effect that occurs in essays that do not exceed a certain length and content, as seen in other studies (Olmos et al, 2009). The effect in …”

Section: Lsa and Human Grader Correlationsmentioning

confidence: 93%

“…More recent studies have questioned the ability of the cosine to measure summaries, with high similarity possibly denoting only paraphrasing and repetition of words (Millis et al, 2004;Kurby et al, 2003;Wolfe & Goldman, 2003). Other studies have demonstrated that Euclidean distances (Formula (6)) have some advantages over cosines in the evaluation of academic essays (Olmos et al, 2009) in essays written by non experts.…”

Section: Cosðvw1; Vw2þmentioning

confidence: 99%

See 1 more Smart Citation

Latent Semantic Analysis Parameters for Essay Evaluation using Small-Scale Corpora*

Jorge-Botana

Gascón

Olmos

et al. 2010

Journal of Quantitative Linguistics

Self Cite

View full text Add to dashboard Cite

Some previous studies (e.g. that carried out by Van Bruggen et al. in 2004) have pointed to a need for additional research in order to firmly establish the usefulness of LSA (latent semantic analysis) parameters for automatic evaluation of academic essays. The extreme variability in approaches to this technique makes it difficult to identify the most efficient parameters and the optimum combination. With this goal in mind, we conducted a high spectrum study to investigate the efficiency of some of the major LSA parameters in small-scale corpora. We used two specific domain corpora that differed in the structure of the text (one containing only technical terms and the other with more tangential information). Using these corpora we tested different semantic spaces, formed by applying different parameters and different methods of comparing the texts. Parameters varied included weighting functions (Log-IDF or Log-Entropy), dimensionality reduction (truncating the matrices after SVD to a set percentage of dimensions), methods of forming pseudo-documents (vector sum and folding-in) and measures of similarity (cosine or Euclidean distances). We also included two groups of essays to be graded, one written by experts and other by non-experts. Both groups were evaluated by three human graders and also by LSA. We extracted the correlations of each LSA condition with human graders, and conducted an ANOVA to analyse which parameter combination correlates best. Results suggest that distances are more efficient in academic essay evaluation than cosines. We found no clear evidence that the classical LSA protocol works systematically better than some simpler version (the classical protocol achieves the best performance only for some combinations of parameters in a few cases), and found that the benefits of reducing dimensionality arise only when the essays are introduced into semantic spaces using the folding-in method.

show abstract

“…Second, because the nomological network must work at the construct level and the construct measurement item level, texts related to these levels should be used to create the aforementioned semantic space. However, LSA semantic spaces based on short text units are known not to have the synonymy and polysemy detection features common in other semantic spaces [37].…”

Section: "A Central Problem a Domain Of Items Taken To Be Facts Relamentioning

confidence: 99%

Developing Interfield Nomological Nets

Larsen

Hovorka

2012

2012 45th Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

New algorithms assessing short summaries in expository texts using latent semantic analysis

Cited by 24 publications

References 18 publications

Reference Production as Search: The Impact of Domain Size on the Production of Distinguishing Descriptions

Reference Production as Search: The Impact of Domain Size on the Production of Distinguishing Descriptions

Latent Semantic Analysis Parameters for Essay Evaluation using Small-Scale Corpora*

Developing Interfield Nomological Nets

Contact Info

Product

Resources

About