OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run git lfs install followed by git lfs pul

Question

I'm using Jupyter Labs on AWS SageMaker.

Kernel: conda_pytorch_p36 and did Restart & Run All.

I git cloned this repo.

Attempt at installing git-lfs:

!curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
!sudo yum install git-lfs -y
!git lfs install

Running fit lfs fetch or git lfs pull after doesn't change Traceback.

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained('albert-base-v2-MRPC')

Traceback:

---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1363                 try:
-> 1364                     state_dict = torch.load(resolved_archive_file, map_location="cpu")
   1365                 except Exception as e:

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    592                 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593         return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
    594 

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
    761 
--> 762     magic_number = pickle_module.load(f, **pickle_load_args)
    763     if magic_number != MAGIC_NUMBER:

UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
      2 
      3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir)  # "textattack/albert-base-v2-MRPC"
      5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
      6 model.to(configs.device)

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    439         elif type(config) in cls._model_mapping.keys():
    440             model_class = _get_model_class(config, cls._model_mapping)
--> 441             return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    442         raise ValueError(
    443             f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1368                             if f.read().startswith("version"):
   1369                                 raise OSError(
-> 1370                                     "You seem to have cloned a repository without having git-lfs installed. Please install "
   1371                                     "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
   1372                                     "you cloned."

OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.
---------------------------------------------------------------------------
UnpicklingError                           Traceback (most recent call last)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1363                 try:
-> 1364                     state_dict = torch.load(resolved_archive_file, map_location="cpu")
   1365                 except Exception as e:

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
    592                 return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
--> 593         return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
    594 

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)
    761 
--> 762     magic_number = pickle_module.load(f, **pickle_load_args)
    763     if magic_number != MAGIC_NUMBER:

UnpicklingError: invalid load key, 'v'.

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-15-34a92ef6f41b> in <module>
      2 
      3 # load model
----> 4 model = AutoModelForSequenceClassification.from_pretrained(configs.output_dir)  # "textattack/albert-base-v2-MRPC"
      5 #model = AlbertForSequenceClassification.from_pretrained(configs.output_dir)
      6 model.to(configs.device)

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/auto_factory.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
    439         elif type(config) in cls._model_mapping.keys():
    440             model_class = _get_model_class(config, cls._model_mapping)
--> 441             return model_class.from_pretrained(pretrained_model_name_or_path, *model_args, config=config, **kwargs)
    442         raise ValueError(
    443             f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   1368                             if f.read().startswith("version"):
   1369                                 raise OSError(
-> 1370                                     "You seem to have cloned a repository without having git-lfs installed. Please install "
   1371                                     "git-lfs and run `git lfs install` followed by `git lfs pull` in the folder "
   1372                                     "you cloned."

OSError: You seem to have cloned a repository without having git-lfs installed. Please install git-lfs and run `git lfs install` followed by `git lfs pull` in the folder you cloned.

albert-base-v2-MRPC/

config.json  log.txt  pytorch_model.bin  README.md  special_tokens_map.json  spiece.model  tokenizer_config.json  train_args.json

Please let me know if there's anything else I can add to post.

Answer 1

I've now installed and initialised GIT LFS in cloned folder.

Terminal:

sh-4.2$ git lfs install
Git LFS initialized.
sh-4.2$ git clone https://huggingface.co/textattack/albert-base-v2-MRPC
Cloning into 'albert-base-v2-MRPC'...
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 27 (delta 7), reused 0 (delta 0)
Unpacking objects: 100% (27/27), done.

sh-4.2$ cd albert-base-v2-MRPC/
sh-4.2$ git lfs install
Updated git hooks.
Git LFS initialized.
sh-4.2$

Answer 2

For Windows 10 and above, just download and install this. Then running the command will work.

Github LTS