Image-Based Classification with Different Optimization Methods

Objective

• Implement Gradient Descent, Stochastic Gradient Descent, and ADAM (used tanh(w*x) function).
• Plot graphs of training and testing loss and correct predictions over iterations.
• Understand how different optimization methods work with image data and compare their performance.

Accomplishments

• Implemented optimization methods in C (no external libraries used).
• Used Python for graph plotting for ease of use.
• Developed a UI using the Java Swing library.

How to Use the App

1. First, prepare two separate classes of images and place them in two folders.
   (Tip: You can name them class_1 and class_2. Example folders can be found in src/exec/img/class1 and class2.)
2. Open the app and click the Photos tab (photo icon on the left side).
   
3. Select your image folders by clicking the Folder icon.
   
4. After selecting the images, click the Create icon in the bottom right corner.
5. Return to the Main tab to tune your parameters, then click the Run button to start.
6. You will see the graphs generated by the program.
   

Conclusion

• With proper tuning, you can observe that Stochastic Gradient Descent uses randomly selected samples, while Gradient Descent uses the entire dataset for each update. ADAM incorporates momentum, leading to faster and often more stable convergence.
• Looking at the t-SNE graph, we can see that ADAM generally performs better than the other optimization methods in terms of convergence.