Questions tagged [topic-modeling]

Topic models describe the frequency of topics in documents and text. A "topic" is a group of words which tend to occur together.

topic-modeling
Filter by
Sorted by
Tagged with
1 vote
0 answers
75 views

Error in posterior function when running LDA

I am trying to conduct topic modelling on a dataset. I follow standard procedure, clean the data, tokenize, create a dtm and apply the LDA function (topics <- tidy(my_topic_model, matrix = "...
Abdul Aziz Rajper's user avatar
1 vote
0 answers
191 views

What if I have too many documents labelled in -1 cluster in bertopic?

I'm generating topics using bertopic on multilingual dataset (mainly Russian and English). I'm reducing the number of topics to 140. After generating topics, I'm analyzing its quality using the ...
ApaarBawa's user avatar
0 votes
0 answers
18 views

size of the node reflect the proportion of each topic

Hi, How do I make the size of the node reflect the proportion of each topic? I am using stm package in R. Also, how to get the colours.
user17122732's user avatar
1 vote
2 answers
1k views

BERTopic guided topic modelling returns a ValueError (inhomogenous shape)

I am trying to train a BERTopic model with a seed topic list. However, the model returns a Value Error: ValueError: setting an array element with a sequence. The requested array has an inhomogeneous ...
Helena's user avatar
  • 33
0 votes
1 answer
93 views

Topic Model LDA: Problem with removing of special character

I want to remove the special character — from my simple corpus. Unfortunately, it doesn't work in my case. I tried different variations of gsub. Also, I tried to copy the dash — from my R object. I ...
Frieda's user avatar
  • 1
0 votes
2 answers
2k views

Evaluating a BERTopic model based on classification metrics

I am unable to find a solution to a problem I have with checking coherence scores for my topic models created using BERTopic. I am new to using these methods for NLP and especially new to using Python....
Tim Mooney's user avatar
3 votes
0 answers
250 views

How long does importing bertopic usually take?

I have been trying this code and it has been running for about 24 hours and it sill running. Is this normal or should I be doing something else? from bertopic import BERTopic Device name LAPTOP-...
Ahmed Ayman's user avatar
0 votes
0 answers
29 views

all-MiniLM-L6-v2 no longer available, any ideas? [duplicate]

What happened to some sentence transformers on Hugging Face? Why is all-MiniLM-L6-v2 not available anymore? I want to use it in the BERTopic model but I cannot. Are there any alternative embedding ...
Aslinur Acarlioglu's user avatar
0 votes
0 answers
46 views

prepDocuments Function in R gives invalid times error

So I created a Structrural Topic Model (https://rdrr.io/cran/stm/), a propabilistic model of text classification which incorporates metadata. Now I want to run a permutation test (https://rdrr.io/cran/...
Tephalu 's Name's user avatar
0 votes
0 answers
59 views

How to create a Document Term Matrix in R (using LSA)?

I'm trying to build a document matrix using the LSA package for my research in R. The txt file I'm trying to read contains text from 10,000 tweets, and there is data in there. But loading TDM results ...
Alex 's user avatar
1 vote
1 answer
485 views

Sentence transformers - KeyError:__version__ error: What are some alternatives?

I'm trying to use sentence transformer with all-miniLM-L6-v2 model for topic modeling but getting error as KeyError:version I have checked in the directory about all the downloaded files and it has ...
Xenon's user avatar
  • 11
0 votes
0 answers
77 views

Labelling automatically data with topics extracted with ldaseqmodel

I'm using the ldaseqmodel for dynamic topic modeling. I would like to label which of the topics I have extracted with ldaseqmodel. So I used the ldaseq.doc_topics(doc_number) method to write the code ...
joy's user avatar
  • 1
0 votes
1 answer
126 views

Is there a method to use LDA gensim with TSNE?

I create a gensim LDA model and i want to present clusters words in the same graphs like TSNE : from gensim.models import LdaModel,lsimodel dictionary = Dictionary(all_texts) corpus = [dictionary....
alex Maia's user avatar
0 votes
1 answer
72 views

Get Highest Topic Probability in each document in Gensim LDA

so my case is, i want to know the topic probability in each document. i'm using get_document_topics() in gensim. here's my code tops = ldamodel.get_document_topics(bow_corpus, minimum_probability=0.3, ...
Rei's user avatar
  • 1
3 votes
3 answers
3k views

Trying to visualize topics using pyldavis but it is giving drop error

I am trying to visualize topics using PyLDAVis but the following code is giving error. Not sure what the issue is. import pyLDAvis.gensim_models pyLDAvis.enable_notebook() vis = pyLDAvis....
Prince Modi's user avatar
2 votes
0 answers
471 views

Failed in nopython mode pipeline (step: analyzing bytecode)

I run this code: from bertopic import BERTopic bertopic_mod = BERTopic.load("bertopic_models/bertopic_complete") I get one of the following errors (a different one each time): IndexError: ...
Delaram R's user avatar
0 votes
1 answer
49 views

How to remove HTML line breaks<br />?

I have a dataset of web scraped reviews and unfortunately they contain a lot of the <br \> tags, so after I clean the data (remove stopwords etc.), a lot of single "br" remain in the ...
TobiP's user avatar
  • 1
0 votes
2 answers
1k views

AttributeError: 'CountVectorizer' object has no attribute 'get_feature_names' -- Topic Modeling -- Latent Dirichlet Allocation

I'm trying to follow the example from the link below. https://medium.datadriveninvestor.com/trump-tweets-topic-modeling-using-latent-dirichlet-allocation-e4f93b90b6fe All the code up to this point ...
ASH's user avatar
  • 19.8k
1 vote
1 answer
130 views

How to understand about "Phi value" in gensim LDA model

At the view of document, I want to know the term probability of the topic for each document from the gensim LdaModel. And I got something like this lda_model = LdaModel(corpus, id2word=dictionary, ...
Leila's user avatar
  • 11
1 vote
1 answer
513 views

mask entities with Spacy NER?

I am working on some topic modeling and my data is heavy on locations. I want to mask them so the model doesn't see them as unique words. I can find them with Spacy using NER, and this is how I'm ...
steadynappin's user avatar
0 votes
2 answers
680 views

visualizing LDA model using pyLDAvis

I'm running jupyter notebook in vscode. I have build my LDA model using gensim but when I want to visualize it, it shows nothing. my code is: import pyLDAvis import pyLDAvis.gensim_models pyLDAvis....
Mhmmd's user avatar
  • 1
0 votes
1 answer
175 views

Measure similarity between text documents

I have 10000 questions framed in different ways, and I want to find the top 20 most frequently asked questions. What would be the best method to do so? So far I'm juggling with topic modelling like ...
Quantizer's user avatar
  • 297
0 votes
2 answers
471 views

Topic modeling with word embeddings

I am currently trying to create an LDA model with word embeddings. Here is the code: from gensim.models.ldamodel import LdaModel from gensim.corpora.dictionary import Dictionary from gensim.test.utils ...
Sebastian Kaczmarczyk's user avatar
0 votes
0 answers
54 views

Filtering user interviews using text response in Python

I am analyzing a large data set of user interviews and am using topic modeling as well as sentiment analysis. To make this more useful, I am filtering by text responses, for example looking at ...
Matt Greenfield's user avatar
0 votes
1 answer
204 views

R: Remove words systematically from corpus after processing topic model

I am doing topic modeling with the topicmodels-package and a corpus consisting of three documents. model <- LDA(dat_dtm, method = "VEM", k = 3, control = list(alpha = 0.1)) Output: A ...
Jonnytheriver's user avatar
0 votes
0 answers
141 views

How to evaluate BTM topic modelling in R when changing K, the number of topics to be detected?

I have a set of titles and I need to cluster them in their semantic space. Hello, I am using library(BTM) to cluster a blog's titles into a semantic group. With BTM implemented in R, it is easy to do ...
Valerio Ficcadenti's user avatar
2 votes
1 answer
6k views

how can I solve the error: The 'stop_words' parameter of TfidfVectorizer must be a str among {'english'}, an instance of 'list' or None?

I try to do Topic Modeling (with german stop words and german text) after the explanation from: Albrecht, Jens, Sidharth Ramachandran, und Christian Winkler. Blueprints for text analysis using Python: ...
SebastianS's user avatar
1 vote
1 answer
241 views

Topic Modeling: How to "find thoughts" STM function when topic model was made with a quanteda dfm object?

I am new to topic modeling, so I will do my best to describe the question I have. basically, I am wanting to visually inspect the documents that make up the topics in my topic model. I used the ...
user21282980's user avatar
0 votes
1 answer
24 views

Finding word co-occurrances in R

I have run topic modelling in R. This has returned to me a number of topics and the probability of a word within my dataframe belonging to that topic. For example, topic 7 is 'religion/christianity', ...
Lui Holohan's user avatar
-1 votes
1 answer
427 views

Break down text into units of sense - text segmentation NLP Python

I have a dataframe text column (in french) and I want to split each text into sentences by their meaning ( break down text into units of sense ), any idea how to do it with Python libraries and NLP ...
Paradisum's user avatar
1 vote
2 answers
2k views

pyLDAvis error AttributeError: 'CountVectorizer' object has no attribute 'get_feature_names'

I am conducting topic modelling for a project of mine and struggled into visualizing the results. I think the proceeding is right. Specifically when I run this line vis = pyLDAvis.sklearn.prepare(...
Blue_point225's user avatar
0 votes
1 answer
327 views

'Duplicate' NGram values in topic list created using bertopic

I've set the CountVectorizer to examine bi and trigrams (ngram_range=(1, 3)) . This seems very useful. However, I'm seeing "duplicate" terms e.g.: The terms "justice," "India,...
Varun Sappa's user avatar
0 votes
0 answers
32 views

Relative pruning/word deletion results in error of Structural topic modeling

I am trying to use a structural topic model to look at topic changes in YouTube comments and i am following this website.: https://bookdown.org/valerie_hase/TextasData_HS2021/tutorial-13-topic-...
Heinz's user avatar
  • 1
0 votes
2 answers
852 views

How to get all documents per topic after merging in BERTopic?

I am using BERTopic to generate topics on my dataset. After the initial topics are created, I used hierarchical clustering to identify some topics I considered too specific, so I created a list of ...
Henrique Back's user avatar
0 votes
1 answer
200 views

Gensim: Not able to load the id2word file

I am working on topic inference on a new corpus given a previously derived lda model. I am able to load the model perfectly, while I am not able to load the id2word file to create the corpora....
Shiyu's user avatar
  • 25
2 votes
1 answer
2k views

Topic Modelling Coherence Score:

I'm trying to calculate the coherence score after using BERTopic modelling to discover topics from an input text. I'm facing this error though "unable to interpret topic as either a list of ...
pceccon's user avatar
  • 9,634
0 votes
0 answers
72 views

UnicodeEncodeError: 'charmap' codec can't encode character '\xef' in position 3: character maps to <undefined>

I'm doing gibbs theme modeling,but it gives me an error. I'm using the following code: In[1] import numpy as np import pandas as pd import gibbslda In[2] df = pd.read_csv(f, sep=',',...
poma's user avatar
  • 1
1 vote
1 answer
824 views

Labeling automatically the topics generated by BERTopic

I trained BERTopic and get my topics. I want now to assign automatically labels to those topics. I came accross a framework called Yake. I would like to know if there is a python code to acheive this ...
hajar hajar's user avatar
0 votes
3 answers
1k views

Problem in importing UMAP while importing bertopic

So everything worked fine with my code and then suddenly the hdbscan was not working anymore, than I re-instaled all packages and now I have a problem with umap. I did what was suggested here and in ...
jvqp's user avatar
  • 249
1 vote
0 answers
262 views

How to select number of topics for Latent Dirichlet Allocation Topic-Model

I am new to topic modeling and I came across LDA model, but I am not sure if I am using it good. As far as I read the documentation, parameter called n_components is parameter that represent number of ...
taga's user avatar
  • 3,703
0 votes
1 answer
748 views

pipe() got an unexpected keyword argument 'n_threads'

I am trying to run this code for an LDA Topic Model for free form text responses. The path is referencing the raw text from the reviews. When I run this, the error is TypeError: pipe() got an ...
joeulaa's user avatar
0 votes
1 answer
2k views

Trouble in installing BERTopic's dependency ''bertopic.dimensionality''

I'm trying to run the following code from the BERTopic documentation: from bertopic import BERTopic from bertopic.dimensionality import BaseDimensionalityReduction # Fit BERTopic without actually ...
jvqp's user avatar
  • 249
2 votes
1 answer
1k views

Unable to install top2vec

I'm trying to install top2vec for my topic analysis project. To install, I've used this command pip install top2vec. The installation process starts normally but it ends with this error: Preparing ...
Araf's user avatar
  • 273
0 votes
0 answers
38 views

Why do I get an empty plot?

I'm doing a topic model and I'm using matplotlib to visualize the results. I would like to plot word counts and the weights of keywords in one chart. The dataframe has the data, but the plot is empty. ...
socialscientist90's user avatar
2 votes
1 answer
385 views

BERTopic: pop from empty list IndexError while Inferencing

I have trained a BERTopic model on colab and I am now trying to use it locally I get the IndexError. IndexError: Failed in nopython mode pipeline (step: analyzing bytecode) pop from empty list The ...
Vai's user avatar
  • 179
2 votes
2 answers
64 views

Iterate function across dataframe

I have a dataset containing pre-processed online reviews, each row contains words from online review. I am doing a Latent Dirichlet Allocation process to extract topics from the entire dataframe. Now, ...
Dewani's user avatar
  • 137
1 vote
1 answer
65 views

Error in tm package while topic modelling

I am running into an error while trying to make a corpus object from the tm package in R. The data have been scraped from a website and I have included the full code below so you can run and see how ...
I_like_insights's user avatar
-1 votes
1 answer
103 views

How to bypass default parameter to include a range or better SQL?

EDITED (AGAIN): added tables and two screenshots (one of Google Sheets Chart and another showing mutliple issues in DS) to help demonstrate what I am seeing. Short Version: I have created a parameter ...
Taylor Luczak's user avatar
5 votes
1 answer
610 views

Cast topic modeling outcome to dataframe

I have used BertTopic with KeyBERT to extract some topics from some docs from bertopic import BERTopic topic_model = BERTopic(nr_topics="auto", verbose=True, n_gram_range=(1, 4), ...
xavi's user avatar
  • 80
0 votes
1 answer
53 views

What is the 'n_components' parameter for tweetopic.dmm.DMM class?

I am working on a project topic modeling tweets using the tweetopic Python library. I want to understand what the parameter "n_components" for the tweetopic.dmm.DMM class is. I see in the ...
Ashley Stevens's user avatar

1
2
3 4 5
20