Looping over a list of terms to search for, I need to create a boolean field for each term according to whether or not it is present in a tokenized pandas series.
List of search terms:
terms = ['innovative', 'data', 'rf']
Dataframe:
df = pd.DataFrame(data={'job_description': [['innovative', 'data', 'science'],
['scientist', 'have', 'a', 'masters'],
['database', 'rf', 'innovative'],
['sciencedata', 'data', 'performance']]})
Desired output for df:
job_description innovative data rf
0 [innovative, data, science] True True False
1 [scientist, have, a, masters] False False False
2 [database, rf, innovative] True False True
3 [sciencedata, data, performance] False True False
Criteria:
- Only exact matches should be replaced (for example, flagging for 'rf' should return
True
for 'rf' butFalse
for 'performance') - each search term should get it's own field and be concatenated to the original df
What I've tried: Failed, as it creates a boolean for each term in the series:
df['innovative'] = df['job_description'].explode().str.contains(r'innovative').groupby(level=-1).agg(list)
Failed:
df['innovative'] = df['job_description'].str.contains('innovative').astype(int, errors='ignore')
Failed:
df.loc[df['job_description'].str.contains(terms)] = 1
Failed: I tried implementing what was documented here (See if item in each row of pandas series), but could not adapt it to create new fields or flag properly
Thanks for any help you can provide!