This project presents a comparative study of various gradient-based optimization algorithms applied to a Convolutional Neural Network (CNN) trained on the Fashion MNIST dataset. The goal is to evaluate and rank each optimizer based on performance metrics such as validation loss, accuracy, and F1 score.

The following optimizers were implemented and analyzed:

• SGD with Momentum
• SGD with Nesterov Momentum
• AdaGrad
• RMSProp
• Adam

Fashion MNIST is used as the benchmark dataset. It consists of grayscale 28x28 pixel images across 10 categories of fashion products.

• Train set: 60,000 images
• Test set: 10,000 images
• Classes: T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot

We used a simple and efficient Convolutional Neural Network (CNN) architecture

Adam led the performance across F1 Score and accuracy, making it the best optimizer for this specific setup.

The project includes visual plots comparing:

• Validation Loss
• Accuracy
• F1 Score

Each optimizer’s learning curve is also visualized for deeper insights into convergence behavior.

pip install -r requirements.txt

You can launch the analysis notebook using:

jupyter notebook optimizers.ipynb

Experiments are modularly structured:

• train_model(optimizer_name): Trains model using the specified optimizer.
• evaluate_model(): Computes accuracy and F1 score.
• plot_metrics(): Generates visual comparisons.

• Add more optimizers like AdamW, RAdam, and Lookahead
• Apply learning rate schedulers (e.g., Cosine Annealing, StepLR)
• Evaluate on more complex datasets (e.g., CIFAR-10)

• Fashion MNIST by Zalando Research

• Tigist Wondimneh
• Nahom Senay