'TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]):

I am trying to calculate the Meteor score for the following:

print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], "an apple on this tree"))

However I am getting this error every time and I am not sure how to fix it.

TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]): an apple on this tree

I also tried to put "an apple on this tree" in a list

    from nltk.translate.meteor_score import meteor_score
import nltk 
print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], ["an apple on this tree"]))

but it gave me this error.

TypeError: "reference" expects pre-tokenized reference (Iterable[str]): this is an apple

Solution 1:^[1]

Looking at the library code, it looks like hypothesis should be an iterable. https://www.nltk.org/_modules/nltk/translate/meteor_score.html. The error is coming from:

if isinstance(hypothesis, str):
        raise TypeError(
            f'"hypothesis" expects pre-tokenized hypothesis (Iterable[str]): {hypothesis}'
        )

Try putting "an apple on this tree" in a list.

Solution 2:^[2]

Actually, I believe the right answer for the problem is to tokenize the sentence before calling the function. For example:

    for line in zip(refs, hypos):
        ref = word_tokenize(line[0])
        hypo = word_tokenize(line[1])
        m_score += meteor_score([ref], hypo)

Where ref and hypo is a sentence string.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Tyler Liu
Solution 2	innicoder

'TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]):

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]