Unidimensional Scaling

Mair, Patrick; Leeuw, Jan de

doi:10.1002/9781118445112.stat06462.pub2

Cited by 9 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is unlikely that a complex concept (e.g. fluency or adequacy) can be captured in a single rating (McIver and Carmines, 1981). Furthermore,a single Likert scale often does not provide enough points of discrimination: a single 7-point Likert question has only 7 points to discriminate on, while 5 7-point Likert questions have 5 * 7 = 35 points of discrimination.…”

Section: Number Of Questions and Types Of Scalesmentioning

confidence: 99%

Best practices for the human evaluation of automatically generated text

Lee

Gatt

Miltenburg

et al. 2019

Proceedings of the 12th International Conference on Natural Language Generation

125

View full text Add to dashboard Cite

Currently, there is little agreement as to how Natural Language Generation (NLG) systems should be evaluated, with a particularly high degree of variation in the way that human evaluation is carried out. This paper provides an overview of how human evaluation is currently conducted, and presents a set of best practices, grounded in the literature. With this paper, we hope to contribute to the quality and consistency of human evaluations in NLG.

show abstract

Section: Number Of Questions and Types Of Scalesmentioning

confidence: 99%

Best practices for the human evaluation of automatically generated text

Lee

Gatt

Miltenburg

et al. 2019

Proceedings of the 12th International Conference on Natural Language Generation

125

View full text Add to dashboard Cite

show abstract

“…The use of non-Euclidean metrics raises another challenge, related to inferring MDS representations themselves. There is evidence that it can be computationally difficult to find multidimensional city-block MDS representations (Groenen, Heiser, & Meulman, 1998;Hubert, Arabie, & Hesson-McInnis, 1992), as well as finding unidimensional MDS representations (Mair & Leeuw, 2014). Given that these difficulties stem from basic geometric properties of the MDS representations, it seems likely they will continue to present an issue for Bayesian methods of inference.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Inference for Multidimensional Scaling Representations with Psychologically-Interpretable Metrics

Gronau¹,

Lee²

2020

Preprint

View full text Add to dashboard Cite

Multidimensional scaling (MDS) models represent stimuli as points in a space consisting of a number of psychological dimensions, such that the distance between pairs of points corresponds to the dissimilarity between the stimuli. Two fundamental challenges in inferring MDS representations from data involve inferring the appropriate number of dimensions, and the metric structure of the space used to measure distance. We approach both challenges as Bayesian model-selection problems. Treating MDS as a generative model, we define priors needed for model identifiability under metrics corresponding to psychologically separable and psychologically integral stimulus domains. We then apply a differential evolution Markov-chain Monte Carlo (DE-MCMC) method for parameter inference, and a Warp-III method for model selection. We apply these methods to five previous data sets, which collectively test the ability of the methods to infer an appropriate dimensionality and to infer whether stimuli are psychologically separable or integral. We demonstrate that our methods produce sensible results, but note a number of remaining technical challenges that need to be solved before the method can easily and generally be applied. We also note the theoretical promise of the generative modeling perspective, discussing new and extended models of MDS representation that could be developed.

show abstract

“…The use of non-Euclidean metrics raises another challenge, related to inferring MDS representations themselves. There is evidence that it can be computationally difficult to find multidimensional city-block MDS representations (Groenen et al 1998;Hubert et al 1992), as well as finding unidimensional MDS representations (Mair and Leeuw 2014). Given that these difficulties stem from basic geometric properties of the MDS representations, it seems likely they will continue to present an issue for Bayesian methods of inference.…”

Section: Introductionmentioning

confidence: 99%

Bayesian Inference for Multidimensional Scaling Representations with Psychologically Interpretable Metrics

Gronau

Lee

2020

Comput Brain Behav

View full text Add to dashboard Cite

Multidimensional scaling (MDS) models represent stimuli as points in a space consisting of a number of psychological dimensions, such that the distance between pairs of points corresponds to the dissimilarity between the stimuli. Two fundamental challenges in inferring MDS representations from data involve inferring the appropriate number of dimensions and the metric structure of the space used to measure distance. We approach both challenges as Bayesian modelselection problems. Treating MDS as a generative model, we define priors needed for model identifiability under metrics corresponding to psychologically separable and psychologically integral stimulus domains. We then apply a differential evolution Markov-chain Monte Carlo (DE-MCMC) method for parameter inference, and a Warp-III method for model selection. We apply these methods to five previous data sets, which collectively test the ability of the methods to infer an appropriate dimensionality and to infer whether stimuli are psychologically separable or integral. We demonstrate that our methods produce sensible results, but note a number of remaining technical challenges that need to be solved before the method can easily and generally be applied. We also note the theoretical promise of the generative modeling perspective, discussing new and extended models of MDS representation that could be developed.

show abstract

Unidimensional Scaling

Cited by 9 publications

References 5 publications

Best practices for the human evaluation of automatically generated text

Best practices for the human evaluation of automatically generated text

Bayesian Inference for Multidimensional Scaling Representations with Psychologically-Interpretable Metrics

Bayesian Inference for Multidimensional Scaling Representations with Psychologically Interpretable Metrics

Contact Info

Product

Resources

About