pipeline.config setting to train custom object detection model with keypoints pre-trained model

Question

I have pre-trained model centernet_hg104_512x512_kpts_coco17_tpu-32, created .record files and annotated with keypoints dataset. When I run command:

python model_main_tf2.py --alsologtostderr  --pipeline_config_path=pipelines/keypoints/centernet_hg104_512x512_kpts_coco17_tpu-32.config --model_dir=workspace/training_dir/centernet_hg104_512x512_kpts_coco17_tpu-32/

the error appeared:

TypeError: in user code:

    File "venv/lib/python3.8/site-packages/object_detection/inputs.py", line 887, in transform_and_pad_input_data_fn  *
        tensor_dict = pad_input_data_to_static_shapes(
    File "venv/lib/python3.8/site-packages/object_detection/inputs.py", line 319, in transform_input_data  *
        out_tensor_dict[flds_gt_kpt_weights] = (
    File "venv/lib/python3.8/site-packages/object_detection/core/keypoint_ops.py", line 349, in keypoint_weights_from_visibilities  *
        per_keypoint_weight_mult = tf.ones((1, num_keypoints,), dtype=tf.float32)

    TypeError: Expected int32, but got None of type 'NoneType

In pipeline.config I have the paths to the label maps files and .record files.

I've trained boxes models without any problems, but with keypoints annotations, I didn't found the right solution.

Answer 1

I have created a detailed github repo Custom Keypoint Detection for dataset preparation, model training and inference on Centernet-hourglass104 keypoint detection model based on Tensorflow Object detection API with examples.

The dataset preparation for Keypoint detection model is completely different from object detection. Unfortunately, there are no official documentation from Tensorflow. Therefore I have curated the steps for dataset preparation, pipeline configuration and model training. This repo could help you in training your keypoint detection model on custom dataset.

Any issues related to the project can be raised in the github itself and doubts can be cleared here.