An analysis of weight initialization methods in connection with different activation functions for feedforward neural networks
Author (Corporation)
Publication date
2024
Typ of student thesis
Course of study
Collections
Type
01A - Journal article
Editors
Editor (Corporation)
Supervisor
Parent work
Evolutionary Intelligence
Special issue
DOI of the original publication
Link
Series
Series number
Volume
17
Issue / Number
Pages / Duration
2081–2089
Patent number
Publisher / Publishing institution
Springer
Place of publication / Event location
Berlin
Edition
Version
Programming language
Assignee
Practice partner / Client
Abstract
The selection of weight initialization in an artificial neural network is one of the key aspects and affects the learning speed, convergence rate and correctness of classification by an artificial neural network. In this paper, we investigate the effects of weight initialization in an artificial neural network. Nguyen-Widrow weight initialization, random initialization, and Xavier initialization method are paired with five different activation functions. This paper deals with a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer. The paired combination of weight initialization methods with activation functions are examined and tested and compared based on their best achieved loss rate in training. This work aims to better understand how weight initialization methods in neural networks, in combination with activation functions, affect the learning speed in comparison after a fixed number of training epochs.
Keywords
Subject (DDC)
330 - Wirtschaft
Event
Exhibition start date
Exhibition end date
Conference start date
Conference end date
Date of the last check
ISBN
ISSN
1864-5917
1864-5909
1864-5909
Language
English
Created during FHNW affiliation
Yes
Strategic action fields FHNW
Publication status
Published
Review
Peer review of the complete publication
Open access category
Hybrid
License
Citation
WONG, Kit, Rolf DORNBERGER und Thomas HANNE, 2024. An analysis of weight initialization methods in connection with different activation functions for feedforward neural networks. Evolutionary Intelligence. 2024. Bd. 17, S. 2081–2089. DOI 10.1007/s12065-022-00795-y. Verfügbar unter: https://doi.org/10.26041/fhnw-10906