Category "nlp"

spaCy: Which component of the pre-trained en_core_web_md pipeline contains the morphologizer/morphological analysis?

I need the mentioned pre-trained pipeline to analyze the morphological features of my text. To disable the rest of the modules I don't need in my pipeline to ma

nlpaug wordembeddings model not working PermissionError: [Errno 13] Permission denied: '.'

I am following these docs to try and do random word insertion: https://nlpaug.readthedocs.io/en/latest/augmenter/word/word_embs.html However when simply trying

Why FLAIR does't recognize the entire location name of simple sentence?

I'm tying to to detect simple location with NER algorithm, and I'm getting semi-correct results: from flair.data import Sentence from flair.models import Sequ

For integer/dates values annotated using Prodigy, does the spaCy model learn the range of values as well?

I have a prodigy session set up to annotate certain numeric values in a document for age (ranges from 0 to 100). I am only annotating the number. My question is

After executing the last line i get following error: ValueError: y should be a 1d array, got an array of shape (4457, 2) instead

y = pd.get_dummies(messages['label']) y = y.iloc[:,1].values from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_tes

Type of adapters for machine translation (AdapterHub tutorial)

I'm following this guide which explains how to apply adapters to a model for a binary classification task, and I want to adapt it to a machine translation task.

Transformers model from Hugging-Face throws error that specific classes couldn t be loaded

Hi after running this code below, I get the following error. ValueError: Could not load model facebook/bart-large-mnli with any of the following classes: (<c

Solving "CUDA out of memory" when fine-tuning GPT-2 (HuggingFace)

I get the reoccuring CUDA out of memory error when using the HuggingFace Transformers library to fine-tune a GPT-2 model and can't seem to solve it, despite my

German keyword search - look for every possible combination

I'm working on a project where I define some nouns like Haus, Boot, Kampf, ... and what to detect every version (singular/plurar) and every combination of these

Extract specific DATE entity from an sentence by the use of spaCy, and calculate the relative time

import spacy import en_core_web_sm nlp = en_core_web_sm.load() doc = nlp('I get cough yesterday, and tomorrow I will go to hostipital') for t in doc.ents: i

How to integrate Amazon lex with MS Bot framework?

I would like to integrate Amazon Lex with the Microsoft Bot Framework. There is a default integration between Microsoft LUIS NLP and the Microsoft Bot Framework

Reproducibility issue with PyTorch

I'm running a script with the same seed and I see results are reproduced on consecutive runs but somehow running the same script with the same seed changes the

How to setup LSTM to use n-grams instead of sequence length?

I currently have an LSTM which uses sequence length as input, but this only allows the LSTM to predict when the input length is equal to the used sequence lengt

Extracting education from Text data

I have bunch of text data describing people's eduation. I have already done some basic NLP processing to those text data. An example would be this : XXX receive

RoBERTa classifier: cannot generate single prediction

I have succesfully trained a text emotion classifier fine-tuning a RoBERTa language model, mostly using a helpful notebook found online. Now I am trying to writ

Use the polarity distribution of word to detect the sentiment of new words

I have just started a project in NLP. Suppose I have a graph for each word that shows the polarity distribution of sentiments for that word in different sentenc

Prodigy + Spacy to train dataset

I would like an example of using Prodigy to train a dataset (text file with some named entities). This file would be in Portuguese. The idea would be to train a

Why is my word lemmatization not working as expected?

Hi stackoverflow community! Long-time reader but first-time poster. I'm currently trying my hand at NLP and after reading a few forum posts touching upon this t

Convert from Prodigy's JSONL format for labeled NER to spaCy's training format?

I am new to Prodigy and spaCy as well as CLI coding. I'd like to use Prodigy to label my data for an NER model, and then use spaCy in python to create models.

FastText 0.9.2 - why is recall 'nan'?

I trained a supervised model in FastText using the Python interface and I'm getting weird results for precision and recall. First, I trained a model: model = fa