1

I have a dataframe with columns tweet_text and lang where I am trying to convert the tweet_text when the lang is en which are a list of strings i.e. ['The', 'Squad', 'for'] to an indexed tuple list [(0, 'The'), (1, 'Squad'), (2, 'for')].

index  lang                                         tweet_text
3853     es    ALINEACION Este es nuestro once para el paidazo
4129     es                               fue el leon mas agil
4531     en  ['The', 'winner', 'of', 'GOAL', 'OF', 'THE', '...
6316     en                            ['The', 'Squad', 'for']
8367     en  ['The', 'group', 'stage', 'draw', 'will', 'tak...
8696     en  ['years', 'ago', 'Leo', 'won', 'his', 'second'...
10509    pt                              Os nossos guardaredes
10579    en  ['and', 'have', 'announced', 'their', 'AllNew'...
10606    en  ['TransEnterix', 'almost', 'doubled', 'climbin...
11289    pt                      Dia de Classico Dia de FC Poo

The code:

df['tweet_text'] = df.apply(lambda x: list(enumerate(x['tweet_text'])) if x['lang'] == 'en' else x['tweet_text'], axis = 1)

Current result which turns each character in a string into an indexed tuple:

index  lang                                         tweet_text
3853     es    ALINEACION Este es nuestro once para el paidazo
4129     es                               fue el leon mas agil
4531     en  [(0, [), (1, '), (2, T), (3, h), (4, e), (5, '...
6316     en  [(0, [), (1, '), (2, T), (3, h), (4, e), (5, '...
8367     en  [(0, [), (1, '), (2, T), (3, h), (4, e), (5, '...
8696     en  [(0, [), (1, '), (2, y), (3, e), (4, a), (5, r...
10509    pt                              Os nossos guardaredes
10579    en  [(0, [), (1, '), (2, a), (3, n), (4, d), (5, '...
10606    en  [(0, [), (1, '), (2, T), (3, r), (4, a), (5, n...
11289    pt                      Dia de Classico Dia de FC Poo

Dictionary of the dataframe:

'lang': {3853: 'es', 4129: 'es', 4531: 'en', 6316: 'en', 8367: 'en', 8696: 'en', 10509: 'pt', 10579: 'en', 10606: 'en', 11289: 'pt'}, 'tweet_text': {3853: 'ALINEACION Este es nuestro once para el paidazo', 4129: 'fue el leon mas agil', 4531: "['The', 'winner', 'of', 'GOAL', 'OF', 'THE', 'MONTH']", 6316: "['The', 'Squad', 'for']", 8367: "['The', 'group', 'stage', 'draw', 'will', 'take', 'place', 'on', 'Friday', 'afternoon', 'at', 'the', 'official', 'CAF', 'headquaers', 'in', 'Cairo', 'Egy']", 8696: "['years', 'ago', 'Leo', 'won', 'his', 'second', 'Ballon', 'dOr', 'Xavi', 'and', 'completed', 'the', 'podium', 'at', 'the', 'p']", 10509: 'Os nossos guardaredes', 10579: "['and', 'have', 'announced', 'their', 'AllNew', 'FUTURE', 'Z', 'boot', 'and', 'capsule', 'collection']", 10606: "['TransEnterix', 'almost', 'doubled', 'climbing', 'as', 'much', 'as', '97', 'on', 'Monday', 'with', 'volume', 'almost', '13', 'times', 'normal', 'after', 'being', 'ment']", 11289: 'Dia de Classico Dia de FC Poo'}}

1 Answer 1

1

Here is problem values for en in column tweet_text are not lists, but strings repr of lists. So is necessary converting them to list, here by ast.literal_eval:

import ast

df['tweet_text'] = df.apply(lambda x: list(enumerate(ast.literal_eval(x['tweet_text']))) if x['lang'] == 'en' else x['tweet_text'], axis = 1)
print (df)

      lang                                         tweet_text
3853    es    ALINEACION Este es nuestro once para el paidazo
4129    es                               fue el leon mas agil
4531    en  [(0, The), (1, winner), (2, of), (3, GOAL), (4...
6316    en                   [(0, The), (1, Squad), (2, for)]
8367    en  [(0, The), (1, group), (2, stage), (3, draw), ...
8696    en  [(0, years), (1, ago), (2, Leo), (3, won), (4,...
10509   pt                              Os nossos guardaredes
10579   en  [(0, and), (1, have), (2, announced), (3, thei...
10606   en  [(0, TransEnterix), (1, almost), (2, doubled),...
11289   pt                      Dia de Classico Dia de FC Poo
1
  • 1
    Thanks just realized they were strings and not list.
    – yudhiesh
    Feb 12, 2021 at 10:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.