Classifier’s Gaze
Training an image classifier to be used as a drawing tool
2023
For: Harvard GSD Quantitative Aesthetics
Role: Software Developer
Overview
This project is an exploration into the “directionality of gaze” in alorgithms, and specfically a pre-trained classifier model.We are training the model with datasets of what is consider informal apparrel and formal apparrel. The concept is to create an interactive feedback loop where the user can manipulate what they are wearing to exaggerate certain qualities the classifer deems as “formal” or “informal.”
Classifier Training
Informal Dataset Formal Dataset
Initial Testing:
Searching for highest activation
Random Walk Threshold Drawing Grid Search
Real-time Gadient Descent:
Using noisy masks
Adding noisy mask to see if covered area positively influence the classification score or not.
By generating a noisy mask and overlaying that on top of the orginal image, we are essentially able to “remove” a specific part of the image and see how this removal influenced the activation value of the classifer model. With this logic, we can find the location that negatively influences the actiation value thus the location where it is most important to the classifer.
Alternative ways of interpreting real-time classification