13

I was mining user timeline data with tweepy and, have faced some difficulties in understanding the following:

  1. Is the 'retweeted' and 'truncated' attribute referring to the same thing (i.e., status text beyond 140 characters)?
  2. If not, what is the difference?
  3. I came across a stackoverflow question where someone asked how to retrieve status text which has been 'chopped' due the the length being over 140 characters. It suggested that there is a retweeted attribute in the _json dictionary which will be true if that is the case and the full status text will under status->retweeted_status->text. However, I have not been able to find it and, the only status text was under status->text ending with '...'. Did I get this wrong and if so, how do I get the full text?

Thanks for your help in advance.

4 Answers 4

19

Is the 'retweeted' and 'truncated' attribute referring to the same thing (i.e., status text beyond 140 characters)? If not, what is the difference?

No, tweets can be truncated not only after a retweet (forward), but also after a reply or a mention (see my example below which is not a retweet). If it is the case, 'truncated' will be set to 'True' (and 'retweeted' will be True or False). It is also possible to have 'retweeted' equals to True while 'truncated' being equals to False, if the tweet is not truncated because its size is well below 140 characters.

It suggested that there is a retweeted attribute in the _json dictionary which will be true if that is the case and the full status text will be under status->retweeted_status->text.

This is true only if the tweet is a genuine retweet. Actually the retweeted_status could itself be truncated if it is coming from another truncated tweet. The best way is to use the tweet_mode='extended' parameter in Tweepy to retrieve the full text (unfortunately not documented in the Tweepy doc). For instance:

(not extended)

print api.get_status('862328512405004288')._json['text']

@tousuncotefoot @equipedefrance @CreditAgricole @AntoGriezmann @KMbappe @layvinkurzawa @UmtitiSam J'ai jamais vue d… https://tco/kALZ2ki9Vc

(extended)

print api.get_status('862328512405004288', tweet_mode='extended')._json['full_text']

@tousuncotefoot @equipedefrance @CreditAgricole @AntoGriezmann @KMbappe @layvinkurzawa @UmtitiSam J'ai jamais vue de match de foot et cela ferait un beau cadeau pour mon copain !! 🙏🏻🙏🏻🙏🏻😍😍

1
  • 1
    Thanks for your answer. I have tried this and, it works as you have explained. May 14, 2017 at 22:44
7

Regarding question 3)

Below is a way to access the extended status text from a Tweepy stream listener, given that the tweet is more than 140 characters (in this case the status object will have an extended_tweet dictionary). If the tweet is not more than 140 characters, then just get the status text as usual:

class MyStreamListener(tweepy.StreamListener):
    def on_status(self, status):
        try:
            text = status.extended_tweet["full_text"]
        except AttributeError:
            text = status.text
2
  • 1
    Exactly what I needed. Thank you :) +1 from me. Nov 21, 2017 at 19:56
  • I am happy that it was of use! :)
    – Jea
    Nov 21, 2017 at 21:57
4

I am also troubled due to this problem.

When the tweet is retweeted, there are two "full_text" tags in .json file. The real full_text can be extracted by calling tag of 'retweet_status'. The solution is as below:

for tweet in tweepy.Cursor(api.search, q = "social", tweet_mode='extended',
                           wait_on_rate_limit = True, wait_on_rate_limit_notify = True, include_entities=True).items(10):
    try:
        print tweet.retweet.im_self._json['retweeted_status']['full_text']
    except:
        print tweet.retweet.im_self._json['full_text']

4

The simplest way worked for me is simply get the status and search for the retweeted full text.

status = api.get_status(id, tweet_mode="extended")
try:
    print(status.retweeted_status.full_text)
except AttributeError:  # Not a Retweet
    print(status.full_text)

this works even in search

tweets=[]
for tweet in api.search(q=keyword, lang="en", count=count, tweet_mode='extended'):
    try:
       tweets.append(tweet.retweeted_status.full_text)
    except AttributeError:  # Not a Retweet
       tweets.append(tweet.full_text)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.