Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents
There are three primary objectives of this work; first: to establish a gas concentration map; second: to estimate the point of emission of the gas; and third: to generate a path from any location to the point of emission for UAVs or UGVs. A mountable array of MOX sensors was developed so that the an...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Published: |
Public Library of Science
2024
|
| Subjects: | |
| Online Access: | http://eprints.um.edu.my/45569/ https://doi.org/10.1371/journal.pone.0296969 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| id |
my.um.eprints.45569 |
|---|---|
| record_format |
eprints |
| spelling |
my.um.eprints.455692024-10-29T08:24:02Z http://eprints.um.edu.my/45569/ Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents ul Husnain, Anees Mokhtar, Norrima Shah, Noraisyah Mohamed Dahari, Mahidzal Azmi, Amirul Asyhraff Iwahashi, Masahiro QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering There are three primary objectives of this work; first: to establish a gas concentration map; second: to estimate the point of emission of the gas; and third: to generate a path from any location to the point of emission for UAVs or UGVs. A mountable array of MOX sensors was developed so that the angles and distances among the sensors, alongside sensors data, were utilized to identify the influx of gas plumes. Gas dispersion experiments under indoor conditions were conducted to train machine learning algorithms to collect data at numerous locations and angles. Taguchi's orthogonal arrays for experiment design were used to identify the gas dispersion locations. For the second objective, the data collected after pre-processing was used to train an off-policy, model-free reinforcement learning agent with a Q-learning policy. After finishing the training from the training data set, Q-learning produces a table called the Q-table. The Q-table contains state-action pairs that generate an autonomous path from any point to the source from the testing dataset. The entire process is carried out in an obstacle-free environment, and the whole scheme is designed to be conducted in three modes: search, track, and localize. The hyperparameter combinations of the RL agent were evaluated through trial-and-error technique and it was found that epsilon = 0.9, gamma = 0.9 and alpha = 0.9 was the fastest path generating combination that took 1258.88 seconds for training and 6.2 milliseconds for path generation. Out of 31 unseen scenarios, the trained RL agent generated successful paths for all the 31 scenarios, however, the UAV was able to reach successfully on the gas source in 23 scenarios, producing a success rate of 74.19%. The results paved the way for using reinforcement learning techniques to be used as autonomous path generation of unmanned systems alongside the need to explore and improve the accuracy of the reported results as future works. Public Library of Science 2024-02 Article PeerReviewed ul Husnain, Anees and Mokhtar, Norrima and Shah, Noraisyah Mohamed and Dahari, Mahidzal and Azmi, Amirul Asyhraff and Iwahashi, Masahiro (2024) Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents. PLoS ONE, 19 (2). e0296969. ISSN 1932-6203, DOI https://doi.org/10.1371/journal.pone.0296969 <https://doi.org/10.1371/journal.pone.0296969>. https://doi.org/10.1371/journal.pone.0296969 10.1371/journal.pone.0296969 |
| institution |
Universiti Malaya |
| building |
UM Library |
| collection |
Institutional Repository |
| continent |
Asia |
| country |
Malaysia |
| content_provider |
Universiti Malaya |
| content_source |
UM Research Repository |
| url_provider |
http://eprints.um.edu.my/ |
| topic |
QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering |
| spellingShingle |
QA75 Electronic computers. Computer science TK Electrical engineering. Electronics Nuclear engineering ul Husnain, Anees Mokhtar, Norrima Shah, Noraisyah Mohamed Dahari, Mahidzal Azmi, Amirul Asyhraff Iwahashi, Masahiro Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| description |
There are three primary objectives of this work; first: to establish a gas concentration map; second: to estimate the point of emission of the gas; and third: to generate a path from any location to the point of emission for UAVs or UGVs. A mountable array of MOX sensors was developed so that the angles and distances among the sensors, alongside sensors data, were utilized to identify the influx of gas plumes. Gas dispersion experiments under indoor conditions were conducted to train machine learning algorithms to collect data at numerous locations and angles. Taguchi's orthogonal arrays for experiment design were used to identify the gas dispersion locations. For the second objective, the data collected after pre-processing was used to train an off-policy, model-free reinforcement learning agent with a Q-learning policy. After finishing the training from the training data set, Q-learning produces a table called the Q-table. The Q-table contains state-action pairs that generate an autonomous path from any point to the source from the testing dataset. The entire process is carried out in an obstacle-free environment, and the whole scheme is designed to be conducted in three modes: search, track, and localize. The hyperparameter combinations of the RL agent were evaluated through trial-and-error technique and it was found that epsilon = 0.9, gamma = 0.9 and alpha = 0.9 was the fastest path generating combination that took 1258.88 seconds for training and 6.2 milliseconds for path generation. Out of 31 unseen scenarios, the trained RL agent generated successful paths for all the 31 scenarios, however, the UAV was able to reach successfully on the gas source in 23 scenarios, producing a success rate of 74.19%. The results paved the way for using reinforcement learning techniques to be used as autonomous path generation of unmanned systems alongside the need to explore and improve the accuracy of the reported results as future works. |
| format |
Article |
| author |
ul Husnain, Anees Mokhtar, Norrima Shah, Noraisyah Mohamed Dahari, Mahidzal Azmi, Amirul Asyhraff Iwahashi, Masahiro |
| author_facet |
ul Husnain, Anees Mokhtar, Norrima Shah, Noraisyah Mohamed Dahari, Mahidzal Azmi, Amirul Asyhraff Iwahashi, Masahiro |
| author_sort |
ul Husnain, Anees |
| title |
Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| title_short |
Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| title_full |
Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| title_fullStr |
Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| title_full_unstemmed |
Gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| title_sort |
gas concentration mapping and source localization for environmental monitoring through unmanned aerial systems using model-free reinforcement learning agents |
| publisher |
Public Library of Science |
| publishDate |
2024 |
| url |
http://eprints.um.edu.my/45569/ https://doi.org/10.1371/journal.pone.0296969 |
| _version_ |
1814933245663379456 |
| score |
13.211869 |