Skoči na glavni sadržaj

Izvorni znanstveni članak

https://doi.org/10.24138/jcomss-2023-0124

Spectral Proximal Method and Saliency Matrix for Robust Deep Learning Optimization

Cherng-Liin Yong orcid id orcid.org/0000-0003-0952-5252 ; Universiti Tunku Abdul Rahman, Jalan Sungai Long, Bandar Sungai Long, Kajang, Selangor, Malaysia
Ban-Hoe Kwan ; Universiti Tunku Abdul Rahman, Jalan Sungai Long, Bandar Sungai Long, Kajang, Selangor, Malaysia
Danny-Wee-Kiat Ng orcid id orcid.org/0000-0001-9972-2676
Hong-Seng Sim ; Universiti Tunku Abdul Rahman, Jalan Sungai Long, Bandar Sungai Long, Kajang, Selangor, Malaysia


Puni tekst: engleski pdf 8.193 Kb

str. 113-124

preuzimanja: 19

citiraj


Sažetak

This paper presents a new training optimizer for
deep learning models, called the Spectral Proximal (SP) method
with saliency matrix, that aims to improve their ability to
generalize to new data. Generalization is the measure of how well
a model can perform on data that it has not seen during training.
The SP method addresses a pair of hurdles affecting generalization: the problem of gradient confusion within complex model
architectures and the limited availability of training data. The
key innovation of the SP method is the use of a proximal operator
with a saliency matrix, which adjusts the descent direction based
on the importance of each parameter and avoids overfit issues.
This leads to improved performance on image classification
(MNIST and CIFAR-10) and object detection (YOLOv7) tasks
and better ability to generalize to new data. We conducted a
comprehensive inquiry by performing experiments on various
configurations while controlling for potential confounding factors.
The SP method consistently outperformed the baseline method
based on the results.

Ključne riječi

spectral proximal method; saliency matrix; deep learning; machine vision; optimization algorithm

Hrčak ID:

315205

URI

https://hrcak.srce.hr/315205

Datum izdavanja:

26.2.2024.

Posjeta: 34 *