Staff View: Spectral-based convolutional neural network without multiple spatial-frequency domain switchings

Spectral-based convolutional neural network without multiple spatial-frequency domain switchings

Recent researches have shown that spectral representation provides a significant speed-up in the massive computation workload of convolution operations in the inference (feed-forward) algorithm of Convolutional Neural Networks (CNNs). This approach results in reducing the computational complexity of...

Full description

Saved in:

Bibliographic Details
Main Authors:	Ayat, Sayed Omid, Hani, Mohamed Khalil, Ab. Rahman, Ab. Al-Hadi, Abdellatef, Hamdan
Format:	Article
Published:	Elsevier B. V. 2019
Subjects:	TK Electrical engineering. Electronics Nuclear engineering
Online Access:	http://eprints.utm.my/id/eprint/87452/ http://dx.doi.org/10.1016/j.neucom.2019.06.094
Tags:	Add Tag No Tags, Be the first to tag this record!

id	my.utm.87452
record_format	eprints
spelling	my.utm.874522020-11-08T03:59:59Z http://eprints.utm.my/id/eprint/87452/ Spectral-based convolutional neural network without multiple spatial-frequency domain switchings Ayat, Sayed Omid Hani, Mohamed Khalil Ab. Rahman, Ab. Al-Hadi Abdellatef, Hamdan TK Electrical engineering. Electronics Nuclear engineering Recent researches have shown that spectral representation provides a significant speed-up in the massive computation workload of convolution operations in the inference (feed-forward) algorithm of Convolutional Neural Networks (CNNs). This approach results in reducing the computational complexity of the classification task, which makes spectral-based CNN suitable for implementation on embedded platform that typically has constrained resources. However, a major challenge in this approach is that the mathematical formulation of a nonlinear activation function in spectral (frequency) domain is currently not available; hence, computation of the activation functions in each layer has to be performed in the spatial domain. This results in several spatial-frequency domain switchings that are computationally very costly, and as such, it would be advantageous to strictly stay in the frequency domain. Hence, in this work, a novel Spectral Rectified Linear Unit (SReLU) for the activation function is proposed, that makes it possible for the computations to remain in the frequency domain, and therefore avoids the multiple compute-intensive domain transformations. To further optimize the classification speed of the network, an efficient spectral-based CNN model is presented that uses only the lower frequency components by way of fusing the convolutional and sub-sampling layers. Additionally, we provide and utilize a frequency domain equivalent of the conventional batch normalization layer that results in improving the accuracy of the network. Experimental results indicate that the proposed spectral-based CNN model achieves up to 17.02 × and 3.45 × faster classification speed (without considerable accuracy loss) on AT&T face recognition and MNIST digit/fashion classification datasets, respectively, as compared to the equivalent models in the spatial domain, hence outperforming conventional approaches significantly. Elsevier B. V. 2019-10 Article PeerReviewed Ayat, Sayed Omid and Hani, Mohamed Khalil and Ab. Rahman, Ab. Al-Hadi and Abdellatef, Hamdan (2019) Spectral-based convolutional neural network without multiple spatial-frequency domain switchings. Neurocomputing, 364 . pp. 152-167. ISSN 0925-2312 http://dx.doi.org/10.1016/j.neucom.2019.06.094
institution	Universiti Teknologi Malaysia
building	UTM Library
collection	Institutional Repository
continent	Asia
country	Malaysia
content_provider	Universiti Teknologi Malaysia
content_source	UTM Institutional Repository
url_provider	http://eprints.utm.my/
topic	TK Electrical engineering. Electronics Nuclear engineering
spellingShingle	TK Electrical engineering. Electronics Nuclear engineering Ayat, Sayed Omid Hani, Mohamed Khalil Ab. Rahman, Ab. Al-Hadi Abdellatef, Hamdan Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
description	Recent researches have shown that spectral representation provides a significant speed-up in the massive computation workload of convolution operations in the inference (feed-forward) algorithm of Convolutional Neural Networks (CNNs). This approach results in reducing the computational complexity of the classification task, which makes spectral-based CNN suitable for implementation on embedded platform that typically has constrained resources. However, a major challenge in this approach is that the mathematical formulation of a nonlinear activation function in spectral (frequency) domain is currently not available; hence, computation of the activation functions in each layer has to be performed in the spatial domain. This results in several spatial-frequency domain switchings that are computationally very costly, and as such, it would be advantageous to strictly stay in the frequency domain. Hence, in this work, a novel Spectral Rectified Linear Unit (SReLU) for the activation function is proposed, that makes it possible for the computations to remain in the frequency domain, and therefore avoids the multiple compute-intensive domain transformations. To further optimize the classification speed of the network, an efficient spectral-based CNN model is presented that uses only the lower frequency components by way of fusing the convolutional and sub-sampling layers. Additionally, we provide and utilize a frequency domain equivalent of the conventional batch normalization layer that results in improving the accuracy of the network. Experimental results indicate that the proposed spectral-based CNN model achieves up to 17.02 × and 3.45 × faster classification speed (without considerable accuracy loss) on AT&T face recognition and MNIST digit/fashion classification datasets, respectively, as compared to the equivalent models in the spatial domain, hence outperforming conventional approaches significantly.
format	Article
author	Ayat, Sayed Omid Hani, Mohamed Khalil Ab. Rahman, Ab. Al-Hadi Abdellatef, Hamdan
author_facet	Ayat, Sayed Omid Hani, Mohamed Khalil Ab. Rahman, Ab. Al-Hadi Abdellatef, Hamdan
author_sort	Ayat, Sayed Omid
title	Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
title_short	Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
title_full	Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
title_fullStr	Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
title_full_unstemmed	Spectral-based convolutional neural network without multiple spatial-frequency domain switchings
title_sort	spectral-based convolutional neural network without multiple spatial-frequency domain switchings
publisher	Elsevier B. V.
publishDate	2019
url	http://eprints.utm.my/id/eprint/87452/ http://dx.doi.org/10.1016/j.neucom.2019.06.094
_version_	1683230769909071872
score	13.251813

Spectral-based convolutional neural network without multiple spatial-frequency domain switchings

Similar Items