Comparative Evaluation of Large Language Models for Automated Text Classification and Data Labeling in Drug Recommendation
1
Issued Date
2025-01-01
Resource Type
Scopus ID
2-s2.0-105040591055
Journal Title
6th Technology Innovation Management and Engineering Science International Conference Times Icon 2025 Proceedings
Rights Holder(s)
SCOPUS
Bibliographic Citation
6th Technology Innovation Management and Engineering Science International Conference Times Icon 2025 Proceedings (2025)
Suggested Citation
Rattanachayabun K., Jamrat S., Samanchuen T. Comparative Evaluation of Large Language Models for Automated Text Classification and Data Labeling in Drug Recommendation. 6th Technology Innovation Management and Engineering Science International Conference Times Icon 2025 Proceedings (2025). doi:10.1109/TIMES-iCON67125.2025.11488078 Retrieved from: https://repository.li.mahidol.ac.th/handle/123456789/117183
Title
Comparative Evaluation of Large Language Models for Automated Text Classification and Data Labeling in Drug Recommendation
Author(s)
Author's Affiliation
Corresponding Author(s)
Other Contributor(s)
Abstract
Data labeling remains one of the most laborintensive stages in artificial intelligence (AI) development, particularly in natural language processing. This study investigates the potential of large language models (LLMs) to automate text classification and reduce the burden of manual annotation. Four state-of-the-art LLMs-GPT-4o, Llama 3.3, Llama 3.1, and Gemma 2-were evaluated on a drug recommendation dataset using three prompting strategies: zero-shot, one-shot, and few-shot learning. Model performance was assessed using F1-scores across therapeutic consideration, use as directed, and contraindication categories. The results indicate that incorporating even a single labeled example substantially improves classification accuracy compared with zero-shot prompting, while the performance gain from one-shot to few-shot prompting is marginal. Among the tested models, Llama 3.3 achieved the most consistent results, whereas Gemma 2 demonstrated strong zeroshot generalization and GPT-4o provided stable cross-strategy performance. The findings highlight the feasibility of employing LLMs for automated data labeling and underscore the efficiency of one-shot prompting as a practical balance between accuracy and computational cost.