An Empirical Analysis of Topic Models: Uncovering the Relationships between Hyperparameters, Document Length and Performance Measures

Terragni, Silvia; Fersini, Elisabetta

doi:10.26615/978-954-452-072-4_157

Cited by 11 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For each model, we optimize the number of topics, ranging from 5 to 100 topics. We select the ranges of the hyper-parameters similarly to previous work (Terragni and Fersini, 2021).…”

Section: Topic Models and Hyper-parameter Settingmentioning

confidence: 99%

“…The framework already provides several features and resources, among which at least 8 topic models, 4 categories of evaluation metrics, and 4 pre-processed datasets. However, the framework uses a single-objective Bayesian optimization approach, disregarding that a user may want to simultaneously optimize more than one objective (Terragni and Fersini, 2021). For example, a user may be interested in obtaining topics that are coherent but also diverse and separated from each other.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Language Transfer for Identifying Diagnostic Paragraphs in Clinical Notes

Liello¹,

Uryupina²,

Moschitti³

2022

Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

View full text Add to dashboard Cite

The eighth edition of the Italian Conference on Computational Linguistics (CLiC-it 2021) was held at Università degli Studi di Milano-Bicocca from 26th to 28th January 2022.After the edition of 2020, which was held in fully virtual mode due to the health emergency related to Covid-19, CLiC-it 2021 represented the first moment for the Italian research community of Computational Linguistics to meet in person after more than one year of full/partial lockdown. Although the conference was held in dual mode, we strongly suggested the participants to attend it coming to Milan. Indeed, we received a strong feedback on this aspect from the community, which was eager to meet in person and enjoy both the scientific and social events together with the colleagues. In total, 99 participants registered to the conference benefiting from the early registration fee, 91 out of which expressed their intention to attend the event in person, which we consider as a very positive indication of enthusiasm from the community, given the uncertain situation due to the evolution of the pandemic in Italy.In total, we received 68 proposals, organized in the following specific tracks: Information Extraction,

show abstract

“…For each model, we optimize the number of topics, ranging from 5 to 100 topics. We select the ranges of the hyper-parameters similarly to previous work (Terragni and Fersini, 2021).…”

Section: Topic Models and Hyper-parameter Settingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Language Transfer for Identifying Diagnostic Paragraphs in Clinical Notes

Liello¹,

Uryupina²,

Moschitti³

2022

Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021

View full text Add to dashboard Cite

show abstract

Automated Topic Exploration in a Cultural Heritage Corpus

Zoutsou,

Sfakakis,

Papachristopoulos

et al. 2024

Metadata and Semantic Research

View full text Add to dashboard Cite

Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels

2023

View full text Add to dashboard Cite

Although the effect of hyperparameters on algorithmic outputs is well known in machine learning, the effects of hyperparameters on information systems that produce user or customer segments are relatively unexplored. This research investigates the effect of varying the number of user segments on the personification of user engagement data in a real analytics information system, employing the concept of persona. We increment the number of personas from 5 to 15 for a total of 330 personas and 33 persona generations. We then examine the effect of changing the hyperparameter on the gender, age, nationality, and combined gender-age-nationality representation of the user population. The results show that despite using the same data and algorithm, varying the number of personas strongly biases the information system’s personification of the user population. The hyperparameter selection for the 990 total personas results in an average deviation of 54.5% for gender, 42.9% for age, 28.9% for nationality, and 40.5% for gender-age-nationality. A repeated analysis of two other organizations shows similar results for all attributes. The deviation occurred for all organizations on all platforms for all attributes, as high as 90.9% in some cases. The results imply that decision makers using analytics information systems should be aware of the effect of hyperparameters on the set of user or customer segments they are exposed to. Organizations looking to effectively use persona analytics systems must be wary that altering the number of personas could substantially change the results, leading to drastically different interpretations about the actual user base.

show abstract

An Empirical Analysis of Topic Models: Uncovering the Relationships between Hyperparameters, Document Length and Performance Measures

Cited by 11 publications

References 14 publications

Language Transfer for Identifying Diagnostic Paragraphs in Clinical Notes

Language Transfer for Identifying Diagnostic Paragraphs in Clinical Notes

Automated Topic Exploration in a Cultural Heritage Corpus

Finetuning Analytics Information Systems for a Better Understanding of Users: Evidence of Personification Bias on Multiple Digital Channels

Contact Info

Product

Resources

About