'How to loop over all examples in a tensorflow DatasetV1Adapter?

I have a dataset of images saved in 97 .tfrecord-files. Each example includes image data as well as groundtruth data (classes and bounding boxes) for object detection. I would like to loop over all the data in the whole dataset to find out how the classes are distributed within the dataset (i.e. create a class histogram). However, I can't figure out how to do that.
I tried the following code to loop over all examples saved in each file, but the inner for-loop unfortunately is an endless loop:

filenames = []
mypath='/home/workspace/data/waymo/training_and_validation/'
for (dirpath, dirnames, fns) in walk(mypath):
    filenames.extend(fns)
    
for tfrecord_file_name in filenames:
    databatch=get_dataset(mypath+tfrecord_file_name)
    print(databatch)
    i=0
    for element in databatch:
        if(i%1000==0):
            print(i)
        i+=1
    break

I also tried to work with tf.compat.v1.data.make_one_shot_iterator() and tf.compat.v1.data.make_initializable_iterator() and their get_next() and get_next_as_optional() methods, but I ran in the same issue.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source