I'm doing some coursework and in this excercise, I'm trying to get the inverse document frequency (idf) from a corpus. Specifically, a dictionary with each entry being {word:idf}
idf is defined as: number of documents in corpus / the number of documents containing the word
I'm running into a problem with the last line of my code below. The first return statement gives a set instead of a dictionary. The second return statement, which is the correct answer, returns a dictionary.
What is causing this difference? What does the colon and the curly bracket operator do here? (In an assignment statement, I thought the colon operator is supposed to just be a simple way of annotating things only?)
def get_idf(corpus):
N = len(corpus)
freq = defaultdict(int)
word = set()
for doc in corpus:
word |= set(doc)
for w in word:
freq[w] = sum([w in c for c in corpus])
#return { N / freq[w] for w in freq }
#vs
#return { w: N/freq[w] for w in freq }
collections.Counter
– Uri Goren Jun 23 '19 at 14:47{'one': 122, 'two': 23}
makes a dict.{}
is an empty dict. This syntax has been around from the start, Use of:
for annotations is newer and not widely used. – hpaulj Jun 23 '19 at 15:15