0

I was trying to only collect tweets in the past 24 hours, so I set since and until, leave the .items() blank, it turned out give me every tweets... Here is my code:

import tweepy 
import pandas as pd


auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)
api = tweepy.API(auth, wait_on_rate_limit=True)



tweets = tweepy.Cursor(api.user_timeline,id=username, since='2020-10-01', until='2020-10-02').items()


tweets_list = [[tweet.text, tweet.created_at, tweet.retweet_count, tweet.favorite_count] for tweet in tweets]
 
#save in a dataframe
tweets_df = pd.DataFrame(tweets_list,columns=['Tweet Text', 'Tweet Datetime', 'Retweets', 'Favorites'])
 
print(tweets_df)

And the print output was like every tweets cuz I did not put a number limitation in the .items()

Anyone could help me with this? Thanks!

2
  • Please provide a minimal reproducible example. Have you done any debugging?
    – AMC
    Commented Oct 4, 2020 at 22:29
  • @AMC Thanks for the reminders, just re-edited it. Yes I've done rebugging, it just doesn't work:(
    – Kidolle
    Commented Oct 7, 2020 at 4:23

1 Answer 1

1

I am facing the same problem as you. The argument sinceand until are not available anymore since version 3.8 of api.user_timeline.

Here the documentation: API user_timeline doc

Solution

The solution to retrieve tweets from a specific day is to extract all the tweets from the user (using a query if you desire) and then filtering the tweets that fulfill the datetime wanted. Moreover, I recommend you to use api.search because it has until argument. Thus, we can ask Twitter for taking tweets before a selected time and then filtering all the tweets that fulfill the datetime, just in case there are tweets older than the day you wanted (older than yesterday in your case).

Here I show you my code:

import tweepy
from requests_oauthlib import OAuth1Session
import datetime

# CREDENTIALS
api_key = "xxxx"
api_secret_key = "xxxx"
access_token = "xxxx"
access_token_secret = "xxxx"

twitter = OAuth1Session(api_key,
                        client_secret=api_secret_key,
                        resource_owner_key=access_token,
                        resource_owner_secret=access_token_secret)
auth = tweepy.OAuthHandler(api_key, api_secret_key)
auth.set_access_token(access_token, access_token_secret)

# setting datetime
today = datetime.datetime.now()
today = today.replace(hour=23, minute=59, second=59, microsecond=999999) # set from the beggining of the day
time_to_the_past = 1 # 1 because we want 1 day before today
yesterday = today - datetime.timedelta(time_to_the_past) 

# Collecting tweets
count = 10 # Set the number of tweets to retrieve
next_day = yesterday + datetime.timedelta(time_to_the_past) # equivalent to today
tweets = tweepy.Cursor(api.search,
                               q="from:maldito_bulo",
                               lang="es",
                               tweet_mode='extended',
                               until = next_day.date() # format YYYY-MM-DD in datetime. Not string. Twitter only extract tweets before that date
                               ).items(count)

# Extracting desired information from each tweet
tweets_list = []
for tweet in tweets:
            # Filtering by date
            if  yesterday.date() == tweet.created_at.date(): # here we take only tweets from the same date as yesterday in format YYYY-MM-DD
                # Accessing to full_text is different between RT and normal tweet
                if 'retweeted_status' in tweet._json:  # if it is a RT take full_text
                    full_text = tweet._json['retweeted_status']['full_text']
                else:  # if it is a normal tweet take the full_text
                    full_text = tweet.full_text

                 tweets_list.append([tweet.user.screen_name,
                                     tweet.id,
                                     full_text,
                                     tweet.user.verified,
                                     str(tweet.created_at.date()),
                                     tweet.user.location,
                                     tweet.user.url,
                                      tweet.entities
                                      ])

Important

  1. Make sure you are searching for tweets with less than a week (7 days old) because Twitter API has this limitation. This is, tweets from more than 7 days ago cannot be extracted. I case you want to extend the date limit you need to upgrade your account to a Premium account.

  2. I have used yesterday as an example, but if you do yesterday = today - datetime.timedelta(x) where x is a number among 0-7 you can extract tweets from just one day among the 7 days Twitter gives you acces. For example, two_days_before_today = today - datetime.timedelta(2) being today = "2020-05-10" will extract tweets only from "2020-03-10".

  3. As far as I have play with the code, I think that until needs to be a datetime object and not a raw string.

I hope you would find it useful for you! Happy coding!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.