Year: 2025
Type: preprint
Abstract: This paper explores the foundations of neural networks by implementing a multi-layer perceptron (MLP) from scratch using only NumPy. The primary objective is to gain a deep understanding of the behavior of activation functions (Sigmoid, ReLU) and loss functions (MSE, Binary Cross-Entropy) in the training process. Experiments were conducted on a binary classification dataset for up to 10,000 epochs. Results reveal that while all combinations of activation and loss functions yield comparable accuracy (95-97%), ReLU combined with Binary Cross-Entropy demonstrates the most stable and consistent convergence.
Author: Viraaj Sharma
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4414520979
Cites:
Cited by:
Related to: