0

I am using the TensorFlow Object Detection API for retraining a COCO-pretrained Faster RCNN Inception v2 model on my custom dataset and recently noticed that several of my models BoxClassifierLoss get worse over the duration of the training (from e.g. 0.17 loss up to 0.38 and after 100 epochs down to 0.24 (thereafter getting worse again or fluctuating without improvement)).

Therefore I am interested in freezing the BoxClassifier to preserve the initial weights that apparently work better.

I read that there is a 'freeze_variables' parameter in the train.proto, but I am unsure as to what variables to freeze exactly.

2 Answers 2

1

Best to my understanding, Vinod's answer is not related to the question asked. If you want to freeze your model to export it, then you can use export_inference_graph. But I understand that what you wish is to freeze variables during training.

As you mentioned yourself, you can specify variables in update_trainable_variables or freeze_variables in order to choose which variables will be trained and which will not. Essentially these are fed to the filter_variables function on your graph in order to choose the variables to include and exclude from training. As can be seen from the description, it expects a pattern using a regular expression. In order to know your variables' names, to include or exclude them - you can inspect your graph. One way to do so, is by using TensorBoard, Graph tab.

On the other hand, I wish to say that this might not be the solution in your case. At the beginning of a training session it is natural to expect high loss or loss increase. However, if after a full training session, the loss fluctuates - then you should inspect the magnitude of the fluctuation. If it's a minor fluctuation, it's natural, if the magnitude is large - then maybe something is wrong in the training configuration. Further analysis of what is going wrong can only be done with more information, e.g. config file, loss graph, data examples, etc.

0

You can freeze model.ckpt meta (checkpoint files) files which are stored in following location:

C:\tensorflow1\models\research\object_detection\training

These checkpoint files are stored frequently during training, so you can check the detail of this file when your error reduces then freeze the same checkpoint to your final model.

For freezing the model, you can use following command:

python export_inference_graph.py --input_type image_tensor --pipeline_config_path training/faster_rcnn_inception_v2_pets.config --trained_checkpoint_prefix training/model.ckpt-XXXX --output_directory inference_graph

Where, XXXX is the number in file name model.ckpt-XXXX.meta. In my case it is model.ckpt-1970.meta, XXXX = 1970.

Checkout my folder structure in the following image.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.