28

I have a list of usernames on Twitter whose profiles are public. I wish to get "all the tweets" they have posted from the day they formed their profile. I checked Twitter4J examples on GitHub.
According to the Twitter API documentation, only the 20 most recent tweets are returned. Is there anyway I could perform my task?

2

3 Answers 3

27

To use Twitter4J to get all posts from a user you'll have to make your request over multiple pages..

Below code based of an example on GitHub

Twitter unauthenticatedTwitter = new TwitterFactory().getInstance();
//First param of Paging() is the page number, second is the number per page (this is capped around 200 I think.
Paging paging = new Paging(1, 100);
List<Status> statuses = unauthenticatedTwitter.getUserTimeline("google",paging);

Just loop and keep grabbing new pages until there are no new posts should work.

4
  • Tyler, Thanks for the response! However, when I run the same code as said above I get this error <code> "[Thu Jun 03 01:29:51 IST 2010]Using class twitter4j.internal.logging.StdOutLoggerFactory as logging factory. [Thu Jun 03 01:29:51 IST 2010]Use twitter4j.internal.http.HttpClientImpl as HttpClient implementation. Failed:Connection refused: connect" </code> I am able to get the public timeline using the getPublicTimeLine() function.
    – Dexter
    Commented Jun 2, 2010 at 7:32
  • I tried the same using authentication twitter instance of my personal ID which is also public, btw. PS: I am sorry for posting the error in such unformatted format. I have tried all HTML tags including <code> and <blockquote> tags.
    – Dexter
    Commented Jun 2, 2010 at 7:41
  • 1
    You shouldn't have to authenticate to do this, are you not able to use the code above to get the public timeline of google or your own page? What are you using for paging? Could you try with just "username" as the only parameter to getUserTimeline()
    – Tyler
    Commented Jun 2, 2010 at 13:37
  • Tyler, Thanks for the response. I am able to retrieve the tweets today! The API is very erratic in it's output as I'm testing the same since the past two days. In any case, The next problem though is that the code works ONLY for the paging parameters 1,100. I tried for different public usernames but in vain. The API limitation is 3200 {16,200} with a count of 200 per request as mentioned on the website. However, I am unable to avail the functionality.
    – Dexter
    Commented Jun 3, 2010 at 6:28
15

Here is how to get ALL tweets for a user (or at least up to ~3200):

import java.util.*;
import twitter4j.*;
import twitter4j.conf.*;

public static void main(String[] a) {

    ConfigurationBuilder cb = new ConfigurationBuilder();
    cb.setOAuthConsumerKey("YOUR KEYS HERE");
    cb.setOAuthConsumerSecret("YOUR KEYS HERE");
    cb.setOAuthAccessToken("YOUR KEYS HERE");
    cb.setOAuthAccessTokenSecret("YOUR KEYS HERE");

    Twitter twitter = new TwitterFactory(cb.build()).getInstance();

    int pageno = 1;
    String user = "cnn";
    List statuses = new ArrayList();

    while (true) {

      try {

        int size = statuses.size(); 
        Paging page = new Paging(pageno++, 100);
        statuses.addAll(twitter.getUserTimeline(user, page));
        if (statuses.size() == size)
          break;
      }
      catch(TwitterException e) {

        e.printStackTrace();
      }
    }

    System.out.println("Total: "+statuses.size());
}
5
  • HI @rednoyz I tried the code but it always give me bad authentication error. 400:The request was invalid. An accompanying error message will explain why. This is the status code will be returned during version 1.0 rate limiting(dev.twitter.com/pages/rate-limiting). In API v1.1, a request without authentication is considered invalid and you will get this response. message - Bad Authentication data. code - 215
    – viren
    Commented Sep 24, 2015 at 4:56
  • @viren did you set your authentication keys? Commented Feb 2, 2016 at 9:46
  • You need to set your authentication keys -- fill in the values in ConfigurationBuilder
    – rednoyz
    Commented Feb 20, 2016 at 10:44
  • You'll only need the consumer keys for this, not the access tokens. little detail.
    – requinard
    Commented Apr 6, 2016 at 12:00
  • I get an error, 'Authentication credentials are missing', unless I use the access tokens as well.
    – rednoyz
    Commented Apr 10, 2016 at 2:44
6

If you read through the Twitter's Documentation, you can retrieve up to 200 tweets at a time if you specify "count=200" in your API request.

You can also use "page=x" to get different paginated results; you could keep doing this until you've retrieved every tweet the user has posted.

I'm not sure how your Java application would create this, but your requests would probably look like:

http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=1
http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=2
http://api.twitter.com/1/statuses/user_timeline.xml?id=SomeUsername&count=200&page=3

... etc.

Keep in mind that these requests are rate-limited so you'd need to be careful not to exceed the limit.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.