'How to loop over all examples in a tensorflow DatasetV1Adapter?
I have a dataset of images saved in 97 .tfrecord
-files. Each example includes image data as well as groundtruth data (classes and bounding boxes) for object detection. I would like to loop over all the data in the whole dataset to find out how the classes are distributed within the dataset (i.e. create a class histogram). However, I can't figure out how to do that.
I tried the following code to loop over all examples saved in each file, but the inner for-loop unfortunately is an endless loop:
filenames = []
mypath='/home/workspace/data/waymo/training_and_validation/'
for (dirpath, dirnames, fns) in walk(mypath):
filenames.extend(fns)
for tfrecord_file_name in filenames:
databatch=get_dataset(mypath+tfrecord_file_name)
print(databatch)
i=0
for element in databatch:
if(i%1000==0):
print(i)
i+=1
break
I also tried to work with tf.compat.v1.data.make_one_shot_iterator()
and tf.compat.v1.data.make_initializable_iterator()
and their get_next()
and get_next_as_optional()
methods, but I ran in the same issue.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|