Comparative Evaluation of Large Language Models for Automated Text Classification and Data Labeling in Drug Recommendation

Abstract

Data labeling remains one of the most laborintensive stages in artificial intelligence (AI) development, particularly in natural language processing. This study investigates the potential of large language models (LLMs) to automate text classification and reduce the burden of manual annotation. Four state-of-the-art LLMs-GPT-4o, Llama 3.3, Llama 3.1, and Gemma 2-were evaluated on a drug recommendation dataset using three prompting strategies: zero-shot, one-shot, and few-shot learning. Model performance was assessed using F1-scores across therapeutic consideration, use as directed, and contraindication categories. The results indicate that incorporating even a single labeled example substantially improves classification accuracy compared with zero-shot prompting, while the performance gain from one-shot to few-shot prompting is marginal. Among the tested models, Llama 3.3 achieved the most consistent results, whereas Gemma 2 demonstrated strong zeroshot generalization and GPT-4o provided stable cross-strategy performance. The findings highlight the feasibility of employing LLMs for automated data labeling and underscore the efficiency of one-shot prompting as a practical balance between accuracy and computational cost.