My dataset is only 10 thousand sentences. I run it in batches of 100, and clear the memory on each run. I manually slice the sentences to only 50 characters. Af
For some reason, I have to use TIMM package offline. But I found that if I use create_model(), for example: self.img_encoder = timm.create_model("swin_base_patc