About backbone frozen training

[ZiYuTou]

Thank you for your outstanding contribution! When reading your code, I found that you froze the backbone (i.e. CLIPVisionTransformer) by setting its learning rate to zero in the settings. However, in the backbone, in addition to the original part of the CLIP, there are also some newly defined fpn layers, including some untrained ConvTranspose2d layers. Do these layers not need to be trained?