0

I have a list of words and I'm trying to turn plural words in singular in python, then I remove the duplicates. This is how I do it :

import spacy
nlp = spacy.load('fr_core_news_md')

words = ['animaux', 'poule', 'adresse', 'animal', 'janvier', 'poules']
clean_words = []

for word in words:
    doc = nlp(word)
    
for token in doc:
    clean_words.append(token.lemma_)
    
clean_words = list(set(clean_words))

This is the output :

['animal', 'janvier', 'poule', 'adresse']

It works well, but my problem is that 'fr_core_news_md' takes a little too long to load so I was wondering if there was another way to do this ?

1 Answer 1

2

The task you trying to do is called lemmatization and it does more than just converting plural to singular, it removes its flexions. It returns the canonical version of a word, the infinitive form of a verb for example.

If you want to use spacy you can make it load quicker by using the disable parameter. For example spacy.load('fr_core_news_md', disable=['parser', 'textcat', 'ner', 'tagger']).

Alternatively, you use treetagger which is kinda hard to install but works great. Or the FrenchLefffLemmatizer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.