Cover Image

Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach

Deep Neural Network (DNN) is a combination method between two different subfields of Machine Learning application, including the Artificial Neural Network (ANN) and Deep Learning (DL). An example of the DNN model is the Attentive Sequence-to-Sequence (Seq2Seq) model that was first created to tackle...

Full description

Saved in:
Bibliographic Details
Main Author: Wan Solehah, Wan Ahmad
Format: Thesis
Language:English
English
English
Published: UNIMAS 2022
Subjects:
T Technology (General)
Online Access:http://ir.unimas.my/id/eprint/43212/6/Final%20Submission%20of%20Thesis%20Form%20-%20Signed.pdf
http://ir.unimas.my/id/eprint/43212/4/MSc%20Thesis_Wan%20Solehah%20Wan%20Ahmad%20%2824pgs%29.pdf
http://ir.unimas.my/id/eprint/43212/8/Wan%20Solehah%20ft.pdf
http://ir.unimas.my/id/eprint/43212/
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.unimas.ir.43212
record_format eprints
spelling my.unimas.ir.432122024-07-16T03:04:19Z http://ir.unimas.my/id/eprint/43212/ Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach Wan Solehah, Wan Ahmad T Technology (General) Deep Neural Network (DNN) is a combination method between two different subfields of Machine Learning application, including the Artificial Neural Network (ANN) and Deep Learning (DL). An example of the DNN model is the Attentive Sequence-to-Sequence (Seq2Seq) model that was first created to tackle a problem setting in language processing. One of the applications is the chatbot model that works explicitly to accurately respond to users' inquiries. Through the years, a chatbot application has seen some improvement, from generating hard-generic responses to more flexible response. The adoption of DNN method into chatbot application produces a new generation chatbot that called as Generative-Based Chatbot. However, it is difficult to create and train a Generative-Based chatbot model that can maintain relevancy of dialogue generation in a long conversation. Hence, this research’s objective aimed to propose an optimization strategy based on Structural Modification and Optimizing Training Network for improving the lacking of accuracy of response in the chatbot application, to propose the algorithm enhancement to improve the current attention mechanism in the Attentive Sequence-to-Sequence model and the network’s training optimization of its inability to memorize the dialogue history, and lastly, to evaluate the accuracy of response of the proposed solution through data training on loss function and real data testing. The structural modification that is based on a slight modification in Additive Attention mechanism. The method is by adding a scaling factor for the dimension of the decoder hidden state. The other one is the network training’s environment optimization that is done through hyperparameter optimization by selecting and fine-tuning high impact parameters which include Optimizer, Learning Rate and Dropout to reduce error rate (loss function). The strategies applied showed that the final accuracy obtained through the training after implementing a modification in the algorithm is at 81% accuracy rate compared to the basic model that recorded its final accuracy at 79% accuracy rate. Meanwhile, after modification and optimization, the model's performance recorded its final value of accuracy and loss rate at 87% and 0.51, respectively. The result indicates the performance of the optimized model outperforms the baseline model. UNIMAS 2022 Thesis NonPeerReviewed text en http://ir.unimas.my/id/eprint/43212/6/Final%20Submission%20of%20Thesis%20Form%20-%20Signed.pdf text en http://ir.unimas.my/id/eprint/43212/4/MSc%20Thesis_Wan%20Solehah%20Wan%20Ahmad%20%2824pgs%29.pdf text en http://ir.unimas.my/id/eprint/43212/8/Wan%20Solehah%20ft.pdf Wan Solehah, Wan Ahmad (2022) Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach. Masters thesis, Universiti Malaysia Sarawak.
institution Universiti Malaysia Sarawak
building Centre for Academic Information Services (CAIS)
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Malaysia Sarawak
content_source UNIMAS Institutional Repository
url_provider http://ir.unimas.my/
language English
English
English
topic T Technology (General)
spellingShingle T Technology (General)
Wan Solehah, Wan Ahmad
Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
description Deep Neural Network (DNN) is a combination method between two different subfields of Machine Learning application, including the Artificial Neural Network (ANN) and Deep Learning (DL). An example of the DNN model is the Attentive Sequence-to-Sequence (Seq2Seq) model that was first created to tackle a problem setting in language processing. One of the applications is the chatbot model that works explicitly to accurately respond to users' inquiries. Through the years, a chatbot application has seen some improvement, from generating hard-generic responses to more flexible response. The adoption of DNN method into chatbot application produces a new generation chatbot that called as Generative-Based Chatbot. However, it is difficult to create and train a Generative-Based chatbot model that can maintain relevancy of dialogue generation in a long conversation. Hence, this research’s objective aimed to propose an optimization strategy based on Structural Modification and Optimizing Training Network for improving the lacking of accuracy of response in the chatbot application, to propose the algorithm enhancement to improve the current attention mechanism in the Attentive Sequence-to-Sequence model and the network’s training optimization of its inability to memorize the dialogue history, and lastly, to evaluate the accuracy of response of the proposed solution through data training on loss function and real data testing. The structural modification that is based on a slight modification in Additive Attention mechanism. The method is by adding a scaling factor for the dimension of the decoder hidden state. The other one is the network training’s environment optimization that is done through hyperparameter optimization by selecting and fine-tuning high impact parameters which include Optimizer, Learning Rate and Dropout to reduce error rate (loss function). The strategies applied showed that the final accuracy obtained through the training after implementing a modification in the algorithm is at 81% accuracy rate compared to the basic model that recorded its final accuracy at 79% accuracy rate. Meanwhile, after modification and optimization, the model's performance recorded its final value of accuracy and loss rate at 87% and 0.51, respectively. The result indicates the performance of the optimized model outperforms the baseline model.
format Thesis
author Wan Solehah, Wan Ahmad
author_facet Wan Solehah, Wan Ahmad
author_sort Wan Solehah, Wan Ahmad
title Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
title_short Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
title_full Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
title_fullStr Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
title_full_unstemmed Improving Attentive Sequence-to-Sequence Generative-Based Chatbot Model Using Deep Neural Network Approach
title_sort improving attentive sequence-to-sequence generative-based chatbot model using deep neural network approach
publisher UNIMAS
publishDate 2022
url http://ir.unimas.my/id/eprint/43212/6/Final%20Submission%20of%20Thesis%20Form%20-%20Signed.pdf
http://ir.unimas.my/id/eprint/43212/4/MSc%20Thesis_Wan%20Solehah%20Wan%20Ahmad%20%2824pgs%29.pdf
http://ir.unimas.my/id/eprint/43212/8/Wan%20Solehah%20ft.pdf
http://ir.unimas.my/id/eprint/43212/
_version_ 1806430311438876672
score 13.211869

Similar Items