Tensorflow/Keras - loading dataset in batches

Question

I'm currently using Keras with Tensorflow backend to train a sequential model. The problem is, that my dataset is pretty large - about 64 thousand images (just under 16GB) for training. Since I have access to a workstation with quite a bit of RAM I'm able to load everything in the memory. However when I try to train the model with model.fit() I'm getting the following error:

tensorflow.python.framework.errors_impl.InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized.

From what I read, the problem should be, that the data cannot be loaded into the GPUs (even though I have four RTX 3080s, 10GB memory). I'm not sure if the issue could lie somewhere else.

In any case, what I was trying to do is, to load the dataset in batches and then train only on parts of the dataset, using the ImageDataGenerator from Keras. However for my application it is important to load the data in a certain way.

My current folder structure looks like this:

training
        -video_1
               -frame_0000.tiff
               -frame_0001.tiff
               ...
               -frame_nnnn.tiff
        -video_2
               -...
        ...
        -video_n

Where every video has a different number of frames. It is important to load just the images from a single video folder and then do some preprocessing, namely convert them into sequences, consisting of 10 frames each. Alternatively I can use a batch size of 10 and load that much from the hard drive. However, it is important, that no sequence contains images, which are not from the same video. I haven't found a way to do this yet, so I wanted to ask if it is possible?

Additionally I would've liked to save model checkpoints from every model, since I need it for some additional evaluation for my thesis. In this relation, I wanted to know whether a single epoch would still mean, that the model is trained on all images?

Cheers