How Large Language Models Enhance Topic Modeling on User-Generated Content
1
Issued Date
2025-01-01
Resource Type
ISSN
17426588
eISSN
17426596
Scopus ID
2-s2.0-105019743703
Journal Title
Journal of Physics Conference Series
Volume
3114
Issue
1
Rights Holder(s)
SCOPUS
Bibliographic Citation
Journal of Physics Conference Series Vol.3114 No.1 (2025)
Suggested Citation
Bui M.P., Nguyen M.T.N. How Large Language Models Enhance Topic Modeling on User-Generated Content. Journal of Physics Conference Series Vol.3114 No.1 (2025). doi:10.1088/1742-6596/3114/1/012011 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/112896
Title
How Large Language Models Enhance Topic Modeling on User-Generated Content
Author(s)
Author's Affiliation
Corresponding Author(s)
Other Contributor(s)
Abstract
Understanding user-generated content (UGC) is crucial for obtaining actionable insights in domains such as e-commerce and hospitality. However, the noisy and redundant nature of such content present challenges for topic modeling methods like Latent Semantic Analysis (LSA). In this paper, we investigate whether preprocessing user reviews with large language models (LLMs) can improve topic modeling performance. Specifically, we compare two input variants: (1) raw reviews and (2) ChatGPT-generated summaries produced via API as concise keyphrases. We apply LSA with varimax rotation on each variant and evaluate the resulting topic models using multiple criteria, including topic coherence (cν), average pairwise Jaccard overlap, and cluster compactness via silhouette scores. Unlike prior work that employs LLMs primarily for post hoc topic labeling or interpretation, our method integrates an LLM directly into the preprocessing pipeline to reshape noisy input into structured, standardized summaries. While ChatGPT-based preprocessing results in lower cν coherence scores likely due to reduced lexical redundancy, it significantly improves topic separation, cluster quality, and topical specificity, leading to more interpretable and well-structured topic models overall.