Classification of User Complaints and Requests in Customer Service Using Bert Fine-Tuned with Lora

Abstract

The categorization of user complaints and requests is an essential task for large companies, as user feedback is crucial for customer satisfaction and business development. However, manual categorization of such data is highly labor intensive and time consuming. To address this, we automated the classification using classical ML methods and BERT on over 18,000 user complaints and requests spanning 11 classes. We utilized LoRA to make the BERT fine-tuning more computationally efficient by reducing trainable parameters while preserving performance. Given the dataset imbalance, we augmented the minority classes with paraphrasing via gemma-7b. The fine-tuned BERT achieved a 5-10% performance improvement over traditional machine learning approaches, including logistic regression, SVM, and XGBoost, and showed robust results exceeding 80% in all key metrics (82% accuracy, 81% precision, 82% recall, and 81% F1-score) on the test set. This work highlights the potential to reduce manual labor costs while ensuring high accuracy in real-world customer service applications