Performance Analysis of Fine-Tuned ChatGPT-3.5 Chatbot for Alni Accessories E-Commerce Service
Main Article Content
Sugiarti
Ihwana As'ad
Al Hilaluddin
Background: The rapid advancement of artificial intelligence (AI) technology, particularly in language modeling, has driven the adoption of chatbots to support e-commerce services. Alni Accessories store faces challenges related to limited product information and slow customer service, which negatively impact user satisfaction.
Aims: This study aims to analyze the performance of a fine-tuned ChatGPT-3.5-based chatbot to improve interaction quality and transaction efficiency on the e-commerce platform.
Methods & Results: The evaluation process was carried out through a series of experiments, where the 13th experiment achieved the best results with a Training Loss of 0.3894 and a Validation Loss of 0.5787. The Training Mean Token Accuracy of 0.8799 and Validation Mean Token Accuracy of 0.7971 indicate the model’s ability to effectively learn language patterns. Furthermore, the Full Validation Loss of 0.6242 and Full Validation Mean Token Accuracy of 0.8122 demonstrate the model’s strong generalization capability. The independent t-test produced a p-value of 0.0001, confirming a statistically significant difference in transaction response time between the old system and the new system. The findings of this study show that the fine-tuned ChatGPT-3.5 chatbot not only accelerates services and improves product information accuracy but also holds great potential for implementation in other sectors such as education and healthcare.
Hairi, M. H., Purnawansyah, P., & Sugiarti, S. (2020). Pencarian Produk Mall Online Menggunakan Metode Multiple Keywords searching. Buletin Sistem Informasi Dan Teknologi Islam, 1(1), 29–35. https://doi.org/10.33096/busiti.v1i1.659
Herfian, M. R., & Adriansyah, A. R. (2021). Analisis dan Perancangan Aplikasi Chatbot dalam Pelayanan Penerimaan Mahasiswa Baru pada Perguruan Tinggi. Jurnal Informatika Terpadu, 7(2), 87–93. https://doi.org/10.54914/jit.v7i2.370
Kappi, C. M. K., & Marlina, L. (2023). The Effect of Chatbot Services on Online Shop Customer Satisfaction. Brilliance: Research of Artificial Intelligence, 3(2), 252–261. https://doi.org/10.47709/brilliance.v3i2.3133
Kishore, Y. (2024). Optimizing Enterprise Conversational AI: Accelerating Response Accuracy with Custom Dataset Fine-Tuning. Intelligent Information Management, 16(02), 65–76. https://doi.org/10.4236/iim.2024.162005
Kole, V., & Sugeng, S. (2021). Implementasi Penjualan Makanan Secara Online dengan Metode DevOps pada Restaurant Zenbu House Jakarta Barat. Jurnal Sosial Teknologi, 1(8), 867–874. https://doi.org/10.59188/jurnalsostech.v1i8.174
Misnawati. (2023). ChatGPT: Keuntungan, Risiko, Dan Penggunaan Bijak Dalam Era Kecerdasan Buatan. PROSIDING SEMINAR NASIONAL PENDIDIKAN BAHASA SASTRA SENI DAN BUDAYA, 54–67.
OpenAI. (2024). Fine-tuning. OpenAI. https://platform.openai.com/docs/guides/fine-tuning?utm_source=chatgpt.com
Septiadi, B, Isnandar, A. (2024). Transformasi Bisnis di Era Digital: Analisis Sistematis Terhadap E-Bisnis di Indonesia pada Konteks UMKM. Journal of Digital Literacy and Volunteering, 2, 38–43. http://jurnal.relawantik.or.id/index.php/ict
Wibowo, S. M., Susanti, E., & Fatkhiyah, E. (2023). Perancangan Aplikasi Mobile Sistem Informasi Akademik Mahasiswa Sebagai Salah Satu Tahapan Metode DevOps. INSOLOGI: Jurnal Sains Dan Teknologi, 2(6), 1191–1202. https://doi.org/10.55123/insologi.v2i6.2876
Xu, N., & Hu, C. (2023). Enhancing E-Commerce Recommendation using Pre-Trained Language Model and Fine-Tuning. http://arxiv.org/abs/2302.04443
