E2v-PhoBERT: A Fine-Tuned PhoBERT Model with Enhanced Accuracy for High-Performance Vietnamese Sentiment Analysis

Abstract

Emojis serve as crucial elements in digital communication, frequently conveying specific emotions such as happiness, sadness, or anger, making them indispensable for precise sentiment analysis. Additionally, they function as auxiliary contextual markers that help disambiguate the intended sentiment of textual messages, thereby mitigating potential misinterpretations. PhoBERT is a widely adopted pre-trained model for Vietnamese language processing due to its effectiveness in various natural language processing (NLP) tasks, including sentiment analysis. However, PhoBERT lacks dedicated emoji processing capabilities, which may limit its performance in tasks that involve sentiment interpretation. To address this limitation, this study proposes a fine-tuning approach for PhoBERT that integrates Emoji2Vec, referred to as E2V-PhoBERT (https://github.com/hqvjet/VivelAI/tree/E2V-PhoBERT). This integration enhances PhoBERT’s ability to process emojis effectively, thereby improving its sentiment analysis capabilities. Experimental evaluations on three benchmark datasets demonstrate that the proposed approach outperforms the previously best-performing method, ViSoBERT, highlighting its effectiveness in Vietnamese sentiment analysis.