Category "nlp"

Extract specific DATE entity from an sentence by the use of spaCy, and calculate the relative time

import spacy import en_core_web_sm nlp = en_core_web_sm.load() doc = nlp('I get cough yesterday, and tomorrow I will go to hostipital') for t in doc.ents: i

How to integrate Amazon lex with MS Bot framework?

I would like to integrate Amazon Lex with the Microsoft Bot Framework. There is a default integration between Microsoft LUIS NLP and the Microsoft Bot Framework

Reproducibility issue with PyTorch

I'm running a script with the same seed and I see results are reproduced on consecutive runs but somehow running the same script with the same seed changes the

How to setup LSTM to use n-grams instead of sequence length?

I currently have an LSTM which uses sequence length as input, but this only allows the LSTM to predict when the input length is equal to the used sequence lengt

Extracting education from Text data

I have bunch of text data describing people's eduation. I have already done some basic NLP processing to those text data. An example would be this : XXX receive

RoBERTa classifier: cannot generate single prediction

I have succesfully trained a text emotion classifier fine-tuning a RoBERTa language model, mostly using a helpful notebook found online. Now I am trying to writ

Use the polarity distribution of word to detect the sentiment of new words

I have just started a project in NLP. Suppose I have a graph for each word that shows the polarity distribution of sentiments for that word in different sentenc

Prodigy + Spacy to train dataset

I would like an example of using Prodigy to train a dataset (text file with some named entities). This file would be in Portuguese. The idea would be to train a

Why is my word lemmatization not working as expected?

Hi stackoverflow community! Long-time reader but first-time poster. I'm currently trying my hand at NLP and after reading a few forum posts touching upon this t

Convert from Prodigy's JSONL format for labeled NER to spaCy's training format?

I am new to Prodigy and spaCy as well as CLI coding. I'd like to use Prodigy to label my data for an NER model, and then use spaCy in python to create models.

FastText 0.9.2 - why is recall 'nan'?

I trained a supervised model in FastText using the Python interface and I'm getting weird results for precision and recall. First, I trained a model: model = fa

How to solve missing words in nltk.corpus.words.words()?

I have tried to remove non-English words from a text. Problem many other words are absent from the NLTK words corpus. My code: import pandas as pd lst = ['

Process and progress for natural language analysis of company communication?

Assume there is a large record of all different kinds of inter-employee and customer communications (e.g. mails, chat transcripts, OCRed letters) which should b

How can I use "NER" for German Language with stanford-corenlp?

I am trying to use nlp for german language but it does not work! I was making the pipeline and then NER to find the entity of each element in sentence which is

'CRF' object has no attribute 'keep_tempfiles'

I have imported ` from itertools import chain import nltk import sklearn import scipy.stats import sklearn_crfsuite from sklearn_crfsuite import scorers,CR

Tokenization of Compound Words not Working in Quanteda

I'm trying to create a dataframe containing specific keywords-in-context using the kwic() function, but unfortunately, I'm running into some error when attempti

How are the TokenEmbeddings in BERT created?

In the paper describing BERT, there is this paragraph about WordPiece Embeddings. We use WordPiece embeddings (Wu et al., 2016) with a 30,000 token vocab

How do I know the order of the classes in a CatBoost classifier weights?

This is a pretty dumb question, but I couldn't find anywhere, so I will take my chances in here... I'm building a classifier using CatBoost. Since this is a NLP

TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]):

I am trying to calculate the Meteor score for the following: print (nltk.translate.meteor_score.meteor_score( ["this is an apple", "that is an apple"], "an

NLP textEmbed function

I am trying to run the textEmbed function in R. Set up needed: require(quanteda) require(quanteda.textstats) require(udpipe) require(reticulate) #udpi