Calibration of Convolutional Neural Networks with Gradual Freezing
Name
Oliver Savolainen
Abstract
Neural networks have been successfully used for various tasks in various fields, with one of the best examples being the use of convolutional neural networks for image classification. However, these models often cannot be trusted, as they are frequently excessively confident in their predictions, assigning too high of a probability to the predicted class.
As a result, it is necessary to calibrate the models. Several calibration methods have been developed to address this problem, one of the most efficient ones is temperature scaling. This study investigates gradual freezing for calibration, specifically for reducing excessive confidence. Gradual freezing refers to a method of training in which the updating of weights for selected layers is stopped at certain moments.
The source code and results of Kängsepp's 2018 master's thesis were used in this work. First, the best ways to implement the method were found based on one model and dataset. Two gradual freezing schemes were chosen, and then several convolutional neural networks were implemented using Kängsepp's work and trained with gradual freezing. The obtained models' results were compared with Kängsepp's results. Although it cannot be claimed that gradual freezing helped reduce excessive confidence based on the metrics used, freezing did reduce the time required for training while still achieving similar results for many models.
As a result, it is necessary to calibrate the models. Several calibration methods have been developed to address this problem, one of the most efficient ones is temperature scaling. This study investigates gradual freezing for calibration, specifically for reducing excessive confidence. Gradual freezing refers to a method of training in which the updating of weights for selected layers is stopped at certain moments.
The source code and results of Kängsepp's 2018 master's thesis were used in this work. First, the best ways to implement the method were found based on one model and dataset. Two gradual freezing schemes were chosen, and then several convolutional neural networks were implemented using Kängsepp's work and trained with gradual freezing. The obtained models' results were compared with Kängsepp's results. Although it cannot be claimed that gradual freezing helped reduce excessive confidence based on the metrics used, freezing did reduce the time required for training while still achieving similar results for many models.
Graduation Thesis language
Estonian
Graduation Thesis type
Bachelor - Computer Science
Supervisor(s)
Meelis Kull
Defence year
2023