Is there a way to convert a list of dict back to tf.data.Dataset?

Question

I used tfds.load to load Cityscapes dataset. The tf.data.Dataset object is enumerable and returns a dict for each enumeration. I added another value to each dict and stored them in a list. I wonder if there's a way to convert this list of dict back to tfds object.

ds=tfds.load('cityscapes',split='train')
batch=ds.take(32)
modified_dicts=[]
for entry in batch: ## entry is <class 'dict'>
    entry['new_value']=get_new_value(entry['image_id']) ## add a custom labeled mask
    modified_dicts.append(entry)

modified_batch=list_to_tfds(modified_dicts) ## I'd like to know what function to use for this step

I have tried to use tf.data.Dataset.from_tensors but it showed this error:

ValueError: Attempt to convert a value ({ ...... }) with an unsupported type (<class 'dict'>) to a Tensor.

Answer 1

I understand you question by you need to modify the dataset values and save it to tfdf load target, you can using dataset and saved where necessary.

Input:

"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: DataSets
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
dataset_cat = tf.data.Dataset.list_files("F:\\datasets\\downloads\\PetImages\\train\\Cat\\*.png")


"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Functions
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
def f1(path, new_size=256) :
    New_path = 'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\03 28x28.jpg'
    New_label = 'Cute'
    return New_path, New_label

list_image = [ 'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\01 28x28.jpg', 
'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\02 28x28.jpg',
'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\03 28x28.jpg',
'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\04 28x28.jpg',
'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\05 28x28.jpg',
'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\06 28x28.jpg']
list_label = [ 'Shopping', 'Bath', 'Shower', 'Sickness', 'Recover', 'Handsome' ]

dataset = tf.data.Dataset.from_tensor_slices((list_image, list_label))
dataset.take(6)

New_dataset = dataset.map(f1)

Output:

for elem in New_dataset.take(6):
    print(elem)
    ### tf.Tensor(b'Cute', shape=(), dtype=string)
    ### (<tf.Tensor: shape=(), dtype=string, numpy=b'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\01 28x28.jpg'>, <tf.Tensor: shape=(), dtype=string, numpy=b'Shopping'>)
    print(elem[0].numpy)
    ### <bound method _EagerTensorBase.numpy of <tf.Tensor: shape=(), dtype=string, numpy=b'C:\\Users\\Jirayu Kaewprateep\\Pictures\\Cats\\samples\\01 28x28.jpg'>>
    element_as_string = str(elem[0].numpy()).split('\'')
    image = plt.imread(os.fspath(element_as_string[1]))
    plt.imshow(image)
    plt.show()
    plt.close()