15

Is it possible to get more than 100 tweets using the Twitter4j API?
If so can anyone point out the way to do so??

1
  • 4
    @NarendraPathai I am..the default counts to 100. I tried using a loop and querying multiple times but I'm getting the same set of 100 tweets repeatedly.. that's why I am asking if there actually is a way? Jul 26, 2013 at 18:03

7 Answers 7

26

Would need to see your code to provide a code example specific to your case, but you can do this through since_id or max_id.

This information is for the Twitter API.

To get the previous 100 tweets:

  1. find the the lowest id in the set that you just retrieved with your query
  2. perform the same query with the max_id option set to the id you just found.

To get the next 100 tweets:

  1. find the the highest id in the set that you just retrieved with your query
  2. perform the same query with the since_id option set to the id you just found.

In Twitter4j, your Query object has two fields that represent the above API options: sinceId and maxId.

1
  • 3
    set max_id to max_id-1, otherwise, you will get the last tweet twice
    – Fabien
    May 19, 2017 at 10:39
4

You can't load more than 100 tweet per request but i don't know why you want this, instead you can load all tweets in "Endless page" i.e loading 10 items each time the user scroll a list .

for example

Query query = new Query("stackoverflow");
query.setCount(10);// sets the number of tweets to return per page, up to a max of 100
QueryResult  result = twitter.search(query);

now if you want to load the next page simple easy

if(result.hasNext())//there is more pages to load
{
query = result.nextQuery();
result = twitter.search(query);
}

and so on.

1
  • This solution would be the most convenient. Unfortunately this solution doesn't seem to work with every query. If your query has special characters, e.g. Query query = new Query("*"); then result.nextQuery() does not correctly encode them leading to an error "could not authenticate you" from the twitter api.
    – asmaier
    Dec 4, 2014 at 12:11
4

Some Java code that iterates to older pages by using lowest id might look like:

Query query = new Query("test");
query.setCount(100);

int searchResultCount;
long lowestTweetId = Long.MAX_VALUE;

do {
    QueryResult queryResult = twitterInstance.search(query);

    searchResultCount = queryResult.getTweets().size();

    for (Status tweet : queryResult.getTweets()) {

        // do whatever with the tweet

        if (tweet.getId() < lowestTweetId) {
            lowestTweetId = tweet.getId();
            query.setMaxId(lowestTweetId);
        }
    }

} while (searchResultCount != 0 && searchResultCount % 100 == 0);
4

Here is how to get ALL tweets for a user (or at least up to ~3200):

import java.util.*;
import twitter4j.*;
import twitter4j.conf.*;

ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("");
cb.setOAuthConsumerSecret("");
cb.setOAuthAccessToken("");
cb.setOAuthAccessTokenSecret("");

Twitter twitter = new TwitterFactory(cb.build()).getInstance();

int pageno = 1;
String user = "cnn";
List statuses = new ArrayList();

while (true) {

  try {

    int size = statuses.size(); 
    Paging page = new Paging(pageno++, 100);
    statuses.addAll(twitter.getUserTimeline(user, page));
    if (statuses.size() == size)
      break;
  }
  catch(TwitterException e) {

    e.printStackTrace();
  }
}

System.out.println("Total: "+statuses.size());
2

To add to Luke's approach Twitter4j does provide pagination to the queries. You can try fetching more than one pages for your query. Set results per page and the page number.

But I suggest first try the since_id and then try pagination.

1

When you get a response containing your first 100 results, you also get the next id with the response. This id can be used as a query parameter "next"= {the id you received from the previous call} while making the call again and it will give you the next 100 tweets.

0

For a given query it is possible to extract more than 100 tweets. For a quick demo you can download a twitter GUI application for tweet extraction available at http://preciselyconcise.com/apis_and_installations/tweets_extraction_from_twitter.php.

By extracting query results from all available pages for that query, you will be able to extract more than 100 tweets if available under that query. I downloaded the GUI application available in that website and i was able to extract more than 1000 tweets for query #happy.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.