Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent

In this paper we propose the fractional gradient descent for increasing the training and work of modern neural networks. This optimizer searches the global minimum of the loss function considering the fractional gradient directions achieved by Riemann-Liouville, Caputo, and Grunwald-Letnikov derivat...

Полное описание

Сохранить в:

Библиографические подробности
Главные авторы:	Abdulkadirov, R. I., Абдулкадиров, Р. И., Lyakhov, P. A., Ляхов, П. А., Baboshina, V. A., Бабошина, В. А., Nagornov, N. N., Нагорнов, Н. Н.
Формат:	Статья
Язык:	English
Опубликовано:	Institute of Electrical and Electronics Engineers Inc. 2024
Темы:	Caputo Stochastic gradient descent Convolutional neural networks Fractional derivatives of Riemann-Liouville Grunwald-Letnikov Multilayer perceptron Optimization algorithms
Online-ссылка:	https://dspace.ncfu.ru/handle/123456789/29336
Метки:	Добавить метку Нет меток, Требуется 1-ая метка записи!

id	ir-123456789-29336
record_format	dspace
spelling	ir-123456789-293362024-12-06T13:27:52Z Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent Abdulkadirov, R. I. Абдулкадиров, Р. И. Lyakhov, P. A. Ляхов, П. А. Baboshina, V. A. Бабошина, В. А. Nagornov, N. N. Нагорнов, Н. Н. Caputo Stochastic gradient descent Convolutional neural networks Fractional derivatives of Riemann-Liouville Grunwald-Letnikov Multilayer perceptron Optimization algorithms In this paper we propose the fractional gradient descent for increasing the training and work of modern neural networks. This optimizer searches the global minimum of the loss function considering the fractional gradient directions achieved by Riemann-Liouville, Caputo, and Grunwald-Letnikov derivatives. The adjusting of size and direction of the fractional gradient, supported by momentum and Nesterov condition, let the proposed optimizer descend into the global minimum of loss functions of neural networks. Utilizing the proposed optimization algorithm in a linear neural network and a visual transformer lets us attain higher accuracy, precision, recall, Macro F1 score by 1.8-4 percentage points than known analogs than state-of-the-art methods in solving pattern recognition problems on images from MNIST and CIFAR10 datasets. Further research of fractional calculus in modern neural network methodology can improve the quality of solving various challenges such as pattern recognition, time series forecasting, moving object detection, and data generation. 2024-12-06T13:26:29Z 2024-12-06T13:26:29Z 2024 Статья Abdulkadirov, R.I., Lyakhov, P.A., Baboshina, V.A., Nagornov, N.N. Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent // IEEE Access. - 2024. - 12. - pp. 168428-168444. - DOI: 10.1109/ACCESS.2024.3491614 https://dspace.ncfu.ru/handle/123456789/29336 en IEEE Access application/pdf application/pdf Institute of Electrical and Electronics Engineers Inc.
institution	СКФУ
collection	Репозиторий
language	English
topic	Caputo Stochastic gradient descent Convolutional neural networks Fractional derivatives of Riemann-Liouville Grunwald-Letnikov Multilayer perceptron Optimization algorithms
spellingShingle	Caputo Stochastic gradient descent Convolutional neural networks Fractional derivatives of Riemann-Liouville Grunwald-Letnikov Multilayer perceptron Optimization algorithms Abdulkadirov, R. I. Абдулкадиров, Р. И. Lyakhov, P. A. Ляхов, П. А. Baboshina, V. A. Бабошина, В. А. Nagornov, N. N. Нагорнов, Н. Н. Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
description	In this paper we propose the fractional gradient descent for increasing the training and work of modern neural networks. This optimizer searches the global minimum of the loss function considering the fractional gradient directions achieved by Riemann-Liouville, Caputo, and Grunwald-Letnikov derivatives. The adjusting of size and direction of the fractional gradient, supported by momentum and Nesterov condition, let the proposed optimizer descend into the global minimum of loss functions of neural networks. Utilizing the proposed optimization algorithm in a linear neural network and a visual transformer lets us attain higher accuracy, precision, recall, Macro F1 score by 1.8-4 percentage points than known analogs than state-of-the-art methods in solving pattern recognition problems on images from MNIST and CIFAR10 datasets. Further research of fractional calculus in modern neural network methodology can improve the quality of solving various challenges such as pattern recognition, time series forecasting, moving object detection, and data generation.
format	Статья
author	Abdulkadirov, R. I. Абдулкадиров, Р. И. Lyakhov, P. A. Ляхов, П. А. Baboshina, V. A. Бабошина, В. А. Nagornov, N. N. Нагорнов, Н. Н.
author_facet	Abdulkadirov, R. I. Абдулкадиров, Р. И. Lyakhov, P. A. Ляхов, П. А. Baboshina, V. A. Бабошина, В. А. Nagornov, N. N. Нагорнов, Н. Н.
author_sort	Abdulkadirov, R. I.
title	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
title_short	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
title_full	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
title_fullStr	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
title_full_unstemmed	Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent
title_sort	improving the accuracy of neural network pattern recognition by fractional gradient descent
publisher	Institute of Electrical and Electronics Engineers Inc.
publishDate	2024
url	https://dspace.ncfu.ru/handle/123456789/29336
work_keys_str_mv	AT abdulkadirovri improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT abdulkadirovri improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT lyakhovpa improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT lâhovpa improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT baboshinava improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT babošinava improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT nagornovnn improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent AT nagornovnn improvingtheaccuracyofneuralnetworkpatternrecognitionbyfractionalgradientdescent
_version_	1842245855413796864

Improving the Accuracy of Neural Network Pattern Recognition by Fractional Gradient Descent

Схожие документы