12

I'm trying to run LDA (Latent Dirichlet Allocation) on a non-English text dataset.

From sklearn's tutorial, there's this part where you count term frequency of the words to feed into the LDA:

tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2,
                            max_features=n_features,
                            stop_words='english')

Which has built-in stop words feature which is only available for English I think. How could I use my own stop words list for this?

1
  • oh my, yeah it worked! should've read the documentation better next time.
    – troll
    Oct 19, 2016 at 7:18

1 Answer 1

23

You may just assign a list of your own words to the stop_words, e.g.:

stop_words = (["word1", "word2","word3"])
2
  • Why a frozenset and not just a list? According to documentation a list is enough Dec 8, 2022 at 21:08
  • 1
    @nivalderramas Yeah, my link does not work now, previously, it showed the source code where frozenset was used. Dec 8, 2022 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.