Questions tagged [nltk]

The Natural Language Toolkit is a Python library for computational linguistics.

Filter by
Sorted by
Tagged with
-2 votes
0 answers
17 views

How can I develop a chatbot with the ability to search and suggest relevant links based on user queries? [closed]

Our team is currently struggling to swiftly locate pertinent documentation web pages for resolving product issues in the absence of immediate senior support. To streamline this process, we're ...
Karthik chary Pathuri's user avatar
0 votes
0 answers
11 views

nltk: parsing raw text into *binary* syntax trees without having to write my own grammar

I'm trying to use the nltk package to parse strings of sentences such that it can separate sentences into a structure like this: Note the binary structure of the tree. This is pretty standard in the ...
Thomas's user avatar
  • 43
1 vote
1 answer
18 views

How to sentence tokenize based on semi colon using nltk

Hi everbody I am new to text processing, so any help is appreciated. I am trying to sentence tokenize based on period AND semi colons. How can I do this using nltk.sent_tokenize? For example if we ...
Ashreen Kaur's user avatar
0 votes
3 answers
40 views

How do I detect if a UTF-8 file uses code-points versus binary encoding?

I have a text file in UTF-8 encoding, but how do I detect if it is in code-points versus binary? I ask because NLTK (https://www.nltk.org/api/nltk.tokenize.html) says I have to decode it from binary, ...
user3213189's user avatar
-3 votes
0 answers
36 views

Implementation of chatbot using spacey [closed]

can we make chatbot for academic website that uses NLU and NLP with spaCy library in python I tried implementing with NLTK abd used TFIDVECTORIZATION and COSINE_ SIMILARITY but the output was not at ...
ALAN C JOHN's user avatar
1 vote
1 answer
36 views

Python nltk RegexpTokenizer with all words and one specific phrase/chunk

Is it possible to redefine a standard nltk.tokenize.RegexpTokenizer to select all words as tokens and allow only one special phrase/chunk? For example, a standard setup would be: from nltk.tokenize ...
striatum's user avatar
  • 1,452
-1 votes
0 answers
28 views

How to align AMR edge labels with PropBank Role Names

So I'm trying to use the Penman library (and possibly some additional ones) to transform an AMR to use the human-readable PropBank role name instead of generic ARG0, ARG1 etc. So for example: (w / ...
Gregory Gelfond's user avatar
1 vote
1 answer
48 views

Language confusion using Open Multilingual Wordnet with NLTK

I'm trying to understand NLTK's Wordnet module. Even downloading the omw-1.4 module, when I request the synsets of a word in Portuguese it returns terms in English. When I ask for the languages loaded ...
Douglas's user avatar
  • 139
0 votes
1 answer
60 views

NLP preprocessing text in Data Frame, what is the correct order?

I’m trying to preprocess a data frame with two columns. Each cell contains a string, called "title" and "body". Based on this article I tried to reproduce the preprocessing. ...
Louis's user avatar
  • 279
0 votes
0 answers
18 views

python project - required packages won't install in pych

currently im bulinding resume analyzer application which is based on python,streamlit, xampp,and nlp I'm getting the error in the terminal - OSError: [E050] Can't find model 'en_core_web_sm'. It ...
Vinamra Dholam's user avatar
0 votes
1 answer
51 views

NLP pre-processing on two columns in data frame gives error

I have the following data frame: gmeDateDf.head(2) title score id url comms_num body timestamp It's not about the money, it's about sending a... 55.0 l6ulcx https://v.redd.it/6j75regs72e61 6.0 NaN ...
Louis's user avatar
  • 279
1 vote
1 answer
50 views

bigram calculation - Memory error, large file problem

Here is a code for bigram calculation from the text corpus: import sys import csv import string import nltk from nltk import word_tokenize from nltk.tokenize import RegexpTokenizer from nltk.util ...
XTRUST.ORG's user avatar
  • 3,330
1 vote
0 answers
18 views

How to use NLTK tokeniser on text with quotation marks?

NLTK's tokeniser is acting very strangely when I try to tokenise a text with quotation marks in it. I have a .txt file like this: This text has some "quotation marks" to really 'make things' ...
AdeDoyle's user avatar
  • 361
-2 votes
1 answer
62 views

Is there a way of programatically extracting terms (length of contract) from free text

I want to extract contract length from text to term in months. Free text fields range from: "2 x 5 year terms", "3 further x 4 years", "two(2) further terms of five(5) years ...
phango's user avatar
  • 3
0 votes
0 answers
16 views

Bug report nltk.translate.bleu_score stopped working on tokens less than or equal to 3

Plat form Windows 11 Anaconda import nltk as nltk nltk.__version__ '3.8.1' The sentence_bleu ought to return 1 for identical translation from nltk.translate.bleu_score import sentence_bleu, ...
ShoutOutAndCalculate's user avatar
0 votes
0 answers
31 views

How can I apply the concordance method (from nlkt) to each line on my Data Frame to find all the contexts which appear a set of words that I defined?

I want to apply the concordance method to each line of messages in my column of messages to figure out the contexts in which these words (study, studies etc.) appear. df['Message_tokenized'] = df['...
npl's user avatar
  • 11
0 votes
1 answer
37 views

Error loading stopwords: <urlopen error [WinError 10060]?

For a sentiment analysis project, I needed to use 'nltk'. import nltk nltk.download('stopwords') but while running this, I keep getting this error, [nltk_data] Error loading stopwords: <urlopen ...
Ajin T's user avatar
  • 11
0 votes
0 answers
15 views

Converting Tokens and Tags to Synsets

I'm trying to write a function to convert a string to a list of synsets. This function should return a list of synsets in a document by first tokenizing the document using nltk.word_tokenize and then ...
Christie's user avatar
1 vote
2 answers
48 views

How to clean a long text from repetitive (duplicate) Paragraphs?

So i have 100000 rows in a dataframe, all containing a text column. which i want to clean before further analysis. i found this answer which gave me a lot of information. However still i have ...
Mostafa Bouzari's user avatar
1 vote
1 answer
99 views

nltk downloading problems - I tried downloading stopwords for nltk toolkit but this error keeps coming [duplicate]

I typed this to download stopwords and then i got this error please help me out Code - import nltk nltk.download('stopwords') Output - [nltk_data] Error loading stopwords: <urlopen error [Errno 54]...
Vaibhavy Raghuvanshi's user avatar
0 votes
0 answers
22 views

Implement chrF++ in NLTK's nltk.translate.chrf_score.sentence_chrf() function

I'm comparing the chrF++ calculation of Evaluate and NLTK library. Here is the evaluate's implementation: import evaluate reference1 = "犯人受到了嚴密的監控。" hypothesis1 = "犯人受到嚴密監視。" chrf ...
Raptor's user avatar
  • 53.4k
0 votes
1 answer
81 views

Return a value from a method and re-use it to the next method inside class

I am trying to understand classes and methods. Let's say we have a custom class called Nlkt_custom with 3 methods allowing me to: read a text file tokenise the text return word occurences Is it a ...
Artemis's user avatar
  • 145
0 votes
1 answer
15 views

NLTK tell if word is generated by CFG

Given string and a context-free grammar I need to tell if the grammar generates the string or not using NLTK I tried using some opensource solution from github but it doesnt work correctly with this ...
rideau's user avatar
  • 1
0 votes
2 answers
125 views

WinError 10060 Connection Timeout Error When Downloading 'punkt' in NLTK

I'm experiencing an issue with the NLTK library in Python, specifically when trying to download the 'punkt' tokenizer models. Here's the error message I receive: Context: I'm attempting to download ...
Shounak Das's user avatar
0 votes
0 answers
24 views

How to print all the NP/VP (noun phrase, verb phrase) and sentence-structure abbreviations in NLTK?

I saw What are all possible POS tags of NLTK? for listing the possible POS tags, but these don't include phrase structure grammar sentence-level tags it seems like VP (verb phrase) and NP (noun phrase)...
Lance's user avatar
  • 76.1k
3 votes
2 answers
80 views

Finding the nouns in a sentence given the context in Python

How to find the nouns in a sentence regarding the context? I am using the nltk library as follows: text = 'I bought a vintage car.' text = nltk.word_tokenize(text) result = nltk.pos_tag(text) result = ...
Shayan's user avatar
  • 387
-1 votes
1 answer
24 views

nltk python library word tokenizatio error [closed]

I am trying to tokenize a file. `AttributeError Traceback (most recent call last) <ipython-input-8-81ae6f78b554> in <cell line: 4>() 2 robert = open('...
i222025 Amna Javaid's user avatar
2 votes
0 answers
2k views

AttributeError: module 'numpy.linalg._umath_linalg' has no attribute '_ilp64'

Running this code block on google colab. "import nltk" is causing the issue. Error Statement: /usr/local/lib/python3.10/dist-packages/numpy/testing/_private/utils.py in <module> ...
Tori's user avatar
  • 31
0 votes
0 answers
26 views

How to solve this error while downloading stopwords using nltk

import nltk nltk.download('stopwords') from nltk.corpus import stopwords stop_words = set(stopwords.words("english")) I run this code in kaggle notebooks. But I am getting following error. [...
Hasitha Gorrepati's user avatar
0 votes
1 answer
37 views

Why Python cannot import functions from utils? (nltk) [duplicate]

I am trying both in Google Collab and Jupyter Notebooks. In Jupyter Notebooks, presents the following error: --------------------------------------------------------------------------- ImportError ...
philosophy's user avatar
0 votes
0 answers
18 views

Lemmas are 'Form' objects

I am trying to find average differences in sentiment for all antonyms in synsets using the function def check_antonym_similar(wnd, senti): """ find the average difference for ...
The Coding Donut's user avatar
-3 votes
3 answers
81 views

Issue installing transformers

I'm doing a NLP project on vscode " amazon reviews sentiment analyzer" every thing is going ok until I reached the part for importing transformers when I'm installing transformers from pip ...
hassan issa's user avatar
0 votes
0 answers
16 views

TypeError: PlaintextCorpusReader.__init__() missing 1 required positional argument: 'fileids'

I'm trying to use the NLTK's PlaintextCorpusReader in Google Colab to read a corpus. I'm practicing now, so my corpus is just one file. I mount Google lab, then read import google.colab.drive google....
Ali's user avatar
  • 1
1 vote
1 answer
41 views

Using a Word Counter in Python is understating results

As a complete preface, I am a beginner and learning. But, here's the sample schema of my products review table. Record_ID Product_ID Review Comment 1234 89847457 I love this product it was shipped ...
user14452102's user avatar
0 votes
0 answers
24 views

Summarize Text / News with some rules in python chatgpt

I am working on project where i have to scrape crypto news and then summarize the news into specfic format, until this everything is working fine , but here comes the tricky part , i have to detect ...
Muhammad Nabeel's user avatar
0 votes
0 answers
26 views

NLTK how to remove common vocabulary, instead of just stop words?

NLTK can be used to remove stopwords. Technique to remove common words(and their plural versions) from a string However, is there a library contain the "common words" such as "near"...
ShoutOutAndCalculate's user avatar
0 votes
0 answers
42 views

How to Add Product ID Search Functionality to Python Word Counter and Display Associated Words?

I have a very large list of Movie Reviews. This is my schema sample...it's about 4000 rows, so there's a lot going on. MovieID FreeformComment 1234 This was a great movie it made me happy 5678 This ...
user14452102's user avatar
1 vote
0 answers
38 views

get unsupported python packages in snowflake snowpark python worksheet

I am using snowflake python worksheet to perform text analysis on some data in a snowflake table. This includes lemmatizing the text I created this function in snowflake python worksheet def ...
G-T's user avatar
  • 11
0 votes
0 answers
49 views

Executing a function on clicking specific Tkinter Label() text

I'm creating a Spellcheck application using Python and Tkinter's ScrolledText(). User input is spellchecked every time the spacebar is hit. The wrongly spelled words are highlighted in red, and a ...
Tushar Rao's user avatar
0 votes
0 answers
24 views

Slug Issue In Deploying Flask App to Heroku

I am currently trying to deploy a Flask application to Heroku but I keep on encountering this error Compiled slug size: 2.3G is too large (max is 500M. I thinking it is from a model that I am ...
yayeey's user avatar
  • 9
0 votes
0 answers
134 views

Couldn't load PUNKT from NLTK

I couldn't load the punkt module from the nltk library in my Jupyter Notebook. An Error occured when I tried to import 'punkt' as: "Error loading punkt: <urlopen error [WinError 10060] A ...
Apurv Mishra's user avatar
0 votes
0 answers
23 views

how to do topic categorization for a string in python

I'm working at a project for which i need to categorize a text as a political news, news and current affairs what libraries do you suggest to do so? i did try doing a cathegorization with keywords, ...
Francesco Serangeli's user avatar
0 votes
2 answers
77 views

Python NLTK text dispersion plot has y vertical axis is in backwards / reversed order

Since last month NLTK dispersion_plot seems to have y (vertical) axis in reversed order on my machine. This is likely something about my versions of software (I am on a school virtual machine). ...
drpawelo's user avatar
  • 2,390
0 votes
0 answers
29 views

Unable to do Pos_tag to extract

For a NLP task, my code is for i in df['token'][1]: wL=nltk.pos_tag(i) Error - "Output exceeds the size limit" while calculating wL. Why am I getting the error instead of the Part of speech ...
Mrinmay's user avatar
1 vote
0 answers
17 views

paragraph to vector spacy

I have a input test string which is multiline input_text = "this is my example text !!. nothing serious about it." I want to create an embedding of fixed length using spacy. How this can be ...
Bharat Sharma's user avatar
0 votes
0 answers
24 views

Using WordNet and the program NLTK on Python, how can I check how many lemmas each language has in WordNet?

Using WordNet and the program NLTK on Python, how can I check how many lemmas each language has in WordNet? I am looking for code that tells me the number of lemmas per language in WordNet. Any help ...
Epictetus135's user avatar
0 votes
0 answers
46 views

how to resolve [nltk_data] Error loading stopwords: <urlopen error [WinError 10060]

Unable to execute the code : nltk.download('stopwords') import nltk #it worked fine nltk.download('stopwords') It's returning the following error message: [nltk_data] Error loading stopwords: <...
Ashutosh Bhoi's user avatar
0 votes
1 answer
62 views

Pre-trained models(Spacy, NLTK etc. ) for Name, Entity, Product, Place recognition for short descriptions

I have very short descriptions of not exceeding 40 characters. I'm using Spacy's NER model to identify Name, Entity, Products and foods. The problem with the text descriptions I have are very short ...
user3437212's user avatar
0 votes
0 answers
47 views

How to match words in a dump to a particular subject

I have 10 words in a text file . - Eg Sarat Saratha Sarathi Saratogue Saravana Saraya Sarbamouli Sarcasm Sarcastic Sarcinae Sarcocystes Sarcodina Sarcoidoins Sarcoma Sarcophaga Sarcophagi ...
sibi kanagaraj's user avatar
0 votes
0 answers
36 views

Calculating conditional entropy bigrams using NLTK and Kneser-Ney smoothing

I am trying to estimate the conditional entropy of a text source at the bigram level. In order to get a good estimate, I need estimates for the probabilities of bigrams. After doing some reading, it ...
chasmani's user avatar
  • 2,382

1
2 3 4 5
143