Questions tagged [nltk]
The Natural Language Toolkit is a Python library for computational linguistics.
7,149
questions
-2
votes
0
answers
17
views
How can I develop a chatbot with the ability to search and suggest relevant links based on user queries? [closed]
Our team is currently struggling to swiftly locate pertinent documentation web pages for resolving product issues in the absence of immediate senior support. To streamline this process, we're ...
0
votes
0
answers
11
views
nltk: parsing raw text into *binary* syntax trees without having to write my own grammar
I'm trying to use the nltk package to parse strings of sentences such that it can separate sentences into a structure like this:
Note the binary structure of the tree. This is pretty standard in the ...
1
vote
1
answer
18
views
How to sentence tokenize based on semi colon using nltk
Hi everbody I am new to text processing, so any help is appreciated. I am trying to sentence tokenize based on period AND semi colons. How can I do this using nltk.sent_tokenize?
For example if we ...
0
votes
3
answers
40
views
How do I detect if a UTF-8 file uses code-points versus binary encoding?
I have a text file in UTF-8 encoding, but how do I detect if it is in code-points versus binary? I ask because NLTK (https://www.nltk.org/api/nltk.tokenize.html) says I have to decode it from binary, ...
-3
votes
0
answers
36
views
Implementation of chatbot using spacey [closed]
can we make chatbot for academic website that uses NLU and NLP with spaCy library in python
I tried implementing with NLTK abd used TFIDVECTORIZATION and COSINE_ SIMILARITY but the output was not at ...
1
vote
1
answer
36
views
Python nltk RegexpTokenizer with all words and one specific phrase/chunk
Is it possible to redefine a standard nltk.tokenize.RegexpTokenizer to select all words as tokens and allow only one special phrase/chunk? For example, a standard setup would be:
from nltk.tokenize ...
-1
votes
0
answers
28
views
How to align AMR edge labels with PropBank Role Names
So I'm trying to use the Penman library (and possibly some additional ones) to transform an AMR to use the human-readable PropBank role name instead of generic ARG0, ARG1 etc. So for example:
(w / ...
1
vote
1
answer
48
views
Language confusion using Open Multilingual Wordnet with NLTK
I'm trying to understand NLTK's Wordnet module. Even downloading the omw-1.4 module, when I request the synsets of a word in Portuguese it returns terms in English.
When I ask for the languages loaded ...
0
votes
1
answer
60
views
NLP preprocessing text in Data Frame, what is the correct order?
I’m trying to preprocess a data frame with two columns. Each cell contains a string, called "title" and "body".
Based on this article I tried to reproduce the preprocessing. ...
0
votes
0
answers
18
views
python project - required packages won't install in pych
currently im bulinding resume analyzer application which is based on python,streamlit, xampp,and nlp
I'm getting the error in the terminal -
OSError: [E050] Can't find model 'en_core_web_sm'. It ...
0
votes
1
answer
51
views
NLP pre-processing on two columns in data frame gives error
I have the following data frame:
gmeDateDf.head(2)
title
score
id
url
comms_num
body
timestamp
It's not about the money, it's about sending a...
55.0
l6ulcx
https://v.redd.it/6j75regs72e61
6.0
NaN
...
1
vote
1
answer
50
views
bigram calculation - Memory error, large file problem
Here is a code for bigram calculation from the text corpus:
import sys
import csv
import string
import nltk
from nltk import word_tokenize
from nltk.tokenize import RegexpTokenizer
from nltk.util ...
1
vote
0
answers
18
views
How to use NLTK tokeniser on text with quotation marks?
NLTK's tokeniser is acting very strangely when I try to tokenise a text with quotation marks in it.
I have a .txt file like this:
This text has some "quotation marks" to really 'make things' ...
-2
votes
1
answer
62
views
Is there a way of programatically extracting terms (length of contract) from free text
I want to extract contract length from text to term in months. Free text fields range from:
"2 x 5 year terms",
"3 further x 4 years",
"two(2) further terms of five(5) years ...
0
votes
0
answers
16
views
Bug report nltk.translate.bleu_score stopped working on tokens less than or equal to 3
Plat form Windows 11 Anaconda
import nltk as nltk
nltk.__version__
'3.8.1'
The sentence_bleu ought to return 1 for identical translation
from nltk.translate.bleu_score import sentence_bleu, ...
0
votes
0
answers
31
views
How can I apply the concordance method (from nlkt) to each line on my Data Frame to find all the contexts which appear a set of words that I defined?
I want to apply the concordance method to each line of messages in my column of messages to figure out the contexts in which these words (study, studies etc.) appear.
df['Message_tokenized'] = df['...
0
votes
1
answer
37
views
Error loading stopwords: <urlopen error [WinError 10060]?
For a sentiment analysis project, I needed to use 'nltk'.
import nltk
nltk.download('stopwords')
but while running this, I keep getting this error,
[nltk_data] Error loading stopwords: <urlopen ...
0
votes
0
answers
15
views
Converting Tokens and Tags to Synsets
I'm trying to write a function to convert a string to a list of synsets. This function should return a list of synsets in a document by first tokenizing the document using nltk.word_tokenize and then ...
1
vote
2
answers
48
views
How to clean a long text from repetitive (duplicate) Paragraphs?
So i have 100000 rows in a dataframe, all containing a text column. which i want to clean before further analysis. i found this answer which gave me a lot of information. However still i have ...
1
vote
1
answer
99
views
nltk downloading problems - I tried downloading stopwords for nltk toolkit but this error keeps coming [duplicate]
I typed this to download stopwords
and then i got this error
please help me out
Code -
import nltk
nltk.download('stopwords')
Output -
[nltk_data] Error loading stopwords: <urlopen error [Errno 54]...
0
votes
0
answers
22
views
Implement chrF++ in NLTK's nltk.translate.chrf_score.sentence_chrf() function
I'm comparing the chrF++ calculation of Evaluate and NLTK library. Here is the evaluate's implementation:
import evaluate
reference1 = "犯人受到了嚴密的監控。"
hypothesis1 = "犯人受到嚴密監視。"
chrf ...
0
votes
1
answer
81
views
Return a value from a method and re-use it to the next method inside class
I am trying to understand classes and methods.
Let's say we have a custom class called Nlkt_custom with 3 methods allowing me to:
read a text file
tokenise the text
return word occurences
Is it a ...
0
votes
1
answer
15
views
NLTK tell if word is generated by CFG
Given string and a context-free grammar I need to tell if the grammar generates the string or not using NLTK
I tried using some opensource solution from github but it doesnt work correctly with this ...
0
votes
2
answers
125
views
WinError 10060 Connection Timeout Error When Downloading 'punkt' in NLTK
I'm experiencing an issue with the NLTK library in Python, specifically when trying to download the 'punkt' tokenizer models. Here's the error message I receive:
Context:
I'm attempting to download ...
0
votes
0
answers
24
views
How to print all the NP/VP (noun phrase, verb phrase) and sentence-structure abbreviations in NLTK?
I saw What are all possible POS tags of NLTK? for listing the possible POS tags, but these don't include phrase structure grammar sentence-level tags it seems like VP (verb phrase) and NP (noun phrase)...
3
votes
2
answers
80
views
Finding the nouns in a sentence given the context in Python
How to find the nouns in a sentence regarding the context? I am using the nltk library as follows:
text = 'I bought a vintage car.'
text = nltk.word_tokenize(text)
result = nltk.pos_tag(text)
result = ...
-1
votes
1
answer
24
views
nltk python library word tokenizatio error [closed]
I am trying to tokenize a file.
`AttributeError Traceback (most recent call last)
<ipython-input-8-81ae6f78b554> in <cell line: 4>()
2 robert = open('...
2
votes
0
answers
2k
views
AttributeError: module 'numpy.linalg._umath_linalg' has no attribute '_ilp64'
Running this code block on google colab. "import nltk" is causing the issue.
Error Statement:
/usr/local/lib/python3.10/dist-packages/numpy/testing/_private/utils.py in <module>
...
0
votes
0
answers
26
views
How to solve this error while downloading stopwords using nltk
import nltk
nltk.download('stopwords')
from nltk.corpus import stopwords
stop_words = set(stopwords.words("english"))
I run this code in kaggle notebooks. But I am getting following error.
[...
0
votes
1
answer
37
views
Why Python cannot import functions from utils? (nltk) [duplicate]
I am trying both in Google Collab and Jupyter Notebooks.
In Jupyter Notebooks, presents the following error:
---------------------------------------------------------------------------
ImportError ...
0
votes
0
answers
18
views
Lemmas are 'Form' objects
I am trying to find average differences in sentiment for all antonyms in synsets using the function
def check_antonym_similar(wnd, senti):
"""
find the average difference for ...
-3
votes
3
answers
81
views
Issue installing transformers
I'm doing a NLP project on vscode " amazon reviews sentiment analyzer" every thing is going ok until I reached the part for importing transformers
when I'm installing transformers from pip ...
0
votes
0
answers
16
views
TypeError: PlaintextCorpusReader.__init__() missing 1 required positional argument: 'fileids'
I'm trying to use the NLTK's PlaintextCorpusReader in Google Colab to read a corpus. I'm practicing now, so my corpus is just one file.
I mount Google lab, then read
import google.colab.drive
google....
1
vote
1
answer
41
views
Using a Word Counter in Python is understating results
As a complete preface, I am a beginner and learning. But, here's the sample schema of my products review table.
Record_ID
Product_ID
Review Comment
1234
89847457
I love this product it was shipped ...
0
votes
0
answers
24
views
Summarize Text / News with some rules in python chatgpt
I am working on project where i have to scrape crypto news and then summarize the news into specfic format, until this everything is working fine , but here comes the tricky part , i have to detect ...
0
votes
0
answers
26
views
NLTK how to remove common vocabulary, instead of just stop words?
NLTK can be used to remove stopwords. Technique to remove common words(and their plural versions) from a string
However, is there a library contain the "common words" such as "near"...
0
votes
0
answers
42
views
How to Add Product ID Search Functionality to Python Word Counter and Display Associated Words?
I have a very large list of Movie Reviews. This is my schema sample...it's about 4000 rows, so there's a lot going on.
MovieID
FreeformComment
1234
This was a great movie it made me happy
5678
This ...
1
vote
0
answers
38
views
get unsupported python packages in snowflake snowpark python worksheet
I am using snowflake python worksheet to perform text analysis on some data in a snowflake table. This includes lemmatizing the text
I created this function in snowflake python worksheet
def ...
0
votes
0
answers
49
views
Executing a function on clicking specific Tkinter Label() text
I'm creating a Spellcheck application using Python and Tkinter's ScrolledText(). User input is spellchecked every time the spacebar is hit. The wrongly spelled words are highlighted in red, and a ...
0
votes
0
answers
24
views
Slug Issue In Deploying Flask App to Heroku
I am currently trying to deploy a Flask application to Heroku but I keep on encountering this error
Compiled slug size: 2.3G is too large (max is 500M. I thinking it is from a model that I am ...
0
votes
0
answers
134
views
Couldn't load PUNKT from NLTK
I couldn't load the punkt module from the nltk library in my Jupyter Notebook.
An Error occured when I tried to import 'punkt' as:
"Error loading punkt: <urlopen error [WinError 10060] A ...
0
votes
0
answers
23
views
how to do topic categorization for a string in python
I'm working at a project for which i need to categorize a text as a political news, news and current affairs
what libraries do you suggest to do so?
i did try doing a cathegorization with keywords, ...
0
votes
2
answers
77
views
Python NLTK text dispersion plot has y vertical axis is in backwards / reversed order
Since last month NLTK dispersion_plot seems to have y (vertical) axis in reversed order on my machine. This is likely something about my versions of software (I am on a school virtual machine).
...
0
votes
0
answers
29
views
Unable to do Pos_tag to extract
For a NLP task,
my code is
for i in df['token'][1]: wL=nltk.pos_tag(i)
Error - "Output exceeds the size limit" while calculating wL.
Why am I getting the error instead of the Part of speech ...
1
vote
0
answers
17
views
paragraph to vector spacy
I have a input test string which is multiline
input_text = "this is my example text !!. nothing serious about it."
I want to create an embedding of fixed length using spacy. How this can be ...
0
votes
0
answers
24
views
Using WordNet and the program NLTK on Python, how can I check how many lemmas each language has in WordNet?
Using WordNet and the program NLTK on Python, how can I check how many lemmas each language has in WordNet? I am looking for code that tells me the number of lemmas per language in WordNet. Any help ...
0
votes
0
answers
46
views
how to resolve [nltk_data] Error loading stopwords: <urlopen error [WinError 10060]
Unable to execute the code :
nltk.download('stopwords')
import nltk #it worked fine
nltk.download('stopwords')
It's returning the following error message:
[nltk_data] Error loading stopwords: <...
0
votes
1
answer
62
views
Pre-trained models(Spacy, NLTK etc. ) for Name, Entity, Product, Place recognition for short descriptions
I have very short descriptions of not exceeding 40 characters. I'm using Spacy's NER model to identify Name, Entity, Products and foods.
The problem with the text descriptions I have are very short ...
0
votes
0
answers
47
views
How to match words in a dump to a particular subject
I have 10 words in a text file . - Eg
Sarat Saratha Sarathi Saratogue Saravana Saraya Sarbamouli Sarcasm
Sarcastic Sarcinae Sarcocystes Sarcodina Sarcoidoins Sarcoma
Sarcophaga Sarcophagi ...
0
votes
0
answers
36
views
Calculating conditional entropy bigrams using NLTK and Kneser-Ney smoothing
I am trying to estimate the conditional entropy of a text source at the bigram level. In order to get a good estimate, I need estimates for the probabilities of bigrams. After doing some reading, it ...