19

How can I avoid circular redirect using HttpClient 4.1.1. As I am getting the error like this:-

executing requestGET http://home.somehost.com/Mynet/pages/cHome.xhtml HTTP/1.1
org.apache.http.client.ClientProtocolException
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:822)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
    at edu.uci.ics.crawler4j.url.WebURL.setURL(WebURL.java:122)
    at edu.uci.ics.crawler4j.crawler.CrawlController.addSeed(CrawlController.java:207)
    at edu.uci.ics.crawler4j.example.advanced.Controller.main(Controller.java:31)
Caused by: org.apache.http.client.CircularRedirectException: Circular redirect to 'http://home.somehost.com/Mynet/pages/Home.xhtml'
    at org.apache.http.impl.client.DefaultRedirectStrategy.getLocationURI(DefaultRedirectStrategy.java:168)
    at org.apache.http.impl.client.DefaultRedirectStrategy.getRedirect(DefaultRedirectStrategy.java:193)
    at org.apache.http.impl.client.DefaultRequestDirector.handleResponse(DefaultRequestDirector.java:1021)
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:482)
    at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)

This is my code...

DefaultHttpClient client = null;

        try
        {
            // Set url
            //URI uri = new URI(url.toString());

            client = new DefaultHttpClient();

            client.getCredentialsProvider().setCredentials(
                    new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM),
                    new UsernamePasswordCredentials("test", "test"));


            URL url1 = new URL (url);
            HttpURLConnection connection = (HttpURLConnection) url1.openConnection();
            connection.setFollowRedirects(false);

            HttpGet request = new HttpGet(url);
            final HttpParams params = new BasicHttpParams();
            HttpClientParams.setRedirecting(params, false);
            HttpContext context = new BasicHttpContext();

            System.out.println("----------------------------------------");
            System.out.println("executing request" + request.getRequestLine());
            HttpResponse response = client.execute(request, context);
            HttpEntity entity = response.getEntity();


            System.out.println(response.getStatusLine());
                    InputStream content = entity.getContent();
                    BufferedReader in   = 
                        new BufferedReader (new InputStreamReader (content));
                    String line;
                    while ((line = in.readLine()) != null) {
                       // System.out.println(line);
                    }
                } catch(Exception e) {
                    e.printStackTrace();
                }
2
  • Are you sure you can avoid it ? If there really is a circular redifection, throwing an exception seems like a reasonable way to indicate that.
    – nos
    Jul 14, 2011 at 18:38
  • @nos Thanks for replying back.. I am getting the same url back for the circular redirection. But If I type that url on the browser then I get three response back from the server with the same url. First is 302 Moved tempo. second is 302 found, third is 200 ok.. with the same url...And I want to grab the content of this url.
    – arsenal
    Jul 14, 2011 at 18:43

6 Answers 6

36

You can set the ClientPNames.ALLOW_CIRCULAR_REDIRECTS to true, this will allow redirects to the same location.

  client.getParams().setParameter(ClientPNames.ALLOW_CIRCULAR_REDIRECTS, true); 

See more info here

6
  • 8
    This is deprecated for current version. Use RequestConfig.custom().setCircularRedirectsAllowed(true).build() instead.
    – keiki
    Apr 18, 2014 at 9:54
  • Actually isn't this just to prevent the HttpClient from throwing redirect error. That means the redirect is still happening ? As Adam has mentioned doesn't this needs a permanent fix at the server ?
    – Ajith M A
    Jun 20, 2014 at 8:20
  • am still getting the following error : Caused by: org.apache.http.client.RedirectException: Maximum redirects (50) exceeded Sep 25, 2014 at 9:47
  • @anonymous Naturally. Your web server is misconfigured, and it has a redirect loop(i.e. even after 50 requests, the server is still redirecting you in this case) That's not something you can fix on the client.
    – nos
    Sep 25, 2014 at 10:17
  • 1
    A browser will send other HTTP headers, such as the User-Agent:, which might make the server behave differently. (e.g. wget on that page is getting 404). If 50 redirects doesn't do it, increasing it is unlikely to fix it - but you can increase it by setting the ClientPNames.MAX_REDIRECTS parameter.
    – nos
    Sep 25, 2014 at 10:45
4

You may try:

RequestConfig requestConfig = RequestConfig.custom()
                              .setCircularRedirectsAllowed(true)
                              .build();

HttpClient httpClient = HttpClients.custom()
                        .setDefaultRequestConfig(requestConfig)
                        .setRedirectStrategy(new LaxRedirectStrategy())
                        .build();

HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory();
requestFactory.setHttpClient(httpClient);
3

You just avoided it. HttpClient detected the circular redirect and threw an exception. Had it not been "avoided", it would continue redirecting forever (until you decided to kill the process). There aren't a whole lot of other options, if that's what the server responds with.

The only way to truly avoid a circular redirect loop is to fix the server.

If you are wondering what is going on (like why it seems to work find in a browser but not from your program), try turning on some of the extra HttpClient logging. In particular, make sure you can see all of the HTTP headers being sent back and forth. You can then look at the conversation taking place when you make the same request in your browser, noting the differences. It could be a missing cookie, crazy browser detection, etc...

There are a number of ways of tracing your browser's communications. Here are a few ways that I often use, in order from easiest to hardest (IMHO):

  • Firefox + HttpFox (or LiveHttpHeaders, Firebug, etc...)
  • Fiddler (Windows only)
  • Wireshark/tcpdump

For low-level testing, try using telnet (unless you use Windows, in which case you may be better off with something like PuTTY/plink) and ruling in/out what changes cause the circular redirects.

4
  • Thanks for replying back.. I am getting the same url back for the circular redirection. But If I type that url on the browser then I get three response back from the server with the same url. First is 302 Moved tempo. second is 302 found, third is 200 ok.. with the same url...And I want to grab the content of this url.
    – arsenal
    Jul 14, 2011 at 18:42
  • I've updated my answer with some suggestions for diagnosing the issue Jul 14, 2011 at 18:54
  • I am using firebug in my firefox.. So in the firebug when I see the response back.. I see 302 Found, 302 Moved temporarily, then atlast 200 OK for the same url in my firebug.
    – arsenal
    Jul 14, 2011 at 18:57
  • I think you should try and look at ALL of the headers, not just the response code, and figure out what it is that is different about the requests from Firefox (that works properly) and the ones from your app (that don't). You can use something like telnet to mix and match headers from the requests Jul 14, 2011 at 19:46
2

There is a bug that will cause circular redirect in Apache HttpClient since 4.0, it wasn't fixed even in the latest version.

In DefaultRequestDirector.java, it creates a HttpRedirect to perform redirection, and it will reuse all headers in your original HttpGet, the problem here is it will also reuse Host header, which mean the server will still get the original host after it's attempt to redirect to new URI.

I fixed this by reimplemented the DefaultRequestDirector:

public class RedirectRequestDirector extends DefaultRequestDirector
{
    RedirectRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler userTokenHandler,
            final HttpParams params) 
    {
        super(requestExec, conman, reustrat, kastrat, rouplan, httpProcessor, retryHandler, redirectHandler, targetAuthHandler, proxyAuthHandler, userTokenHandler, params);

    }
    @Override
    protected RoutedRequest handleResponse(RoutedRequest roureq,
            HttpResponse response,
            HttpContext context)
                    throws HttpException, IOException
    {
        RoutedRequest req = super.handleResponse(roureq, response, context);
        if(req != null)
        {
            String redirectTarget = req.getRoute().getTargetHost().getHostName();
            req.getRequest().getOriginal().setHeader("Host", redirectTarget);
        }
        return req;
    }

}

and DefaultHttpClient:

public class RedirectHttpClient extends DefaultHttpClient
{
    @Override
    protected RequestDirector createClientRequestDirector(
            final HttpRequestExecutor requestExec,
            final ClientConnectionManager conman,
            final ConnectionReuseStrategy reustrat,
            final ConnectionKeepAliveStrategy kastrat,
            final HttpRoutePlanner rouplan,
            final HttpProcessor httpProcessor,
            final HttpRequestRetryHandler retryHandler,
            final RedirectHandler redirectHandler,
            final AuthenticationHandler targetAuthHandler,
            final AuthenticationHandler proxyAuthHandler,
            final UserTokenHandler stateHandler,
            final HttpParams params) {
        return new RedirectRequestDirector(
                requestExec,
                conman,
                reustrat,
                kastrat,
                rouplan,
                httpProcessor,
                retryHandler,
                redirectHandler,
                targetAuthHandler,
                proxyAuthHandler,
                stateHandler,
                params);
    }
}

Now I won't complain about the Circular Redirect.

0

Check that your request isnt sent to a proxy before being sent to the url you requested.

0

I faced this issue while spring version upgrade, the context is not initialized properly in my case.

In org.apache.http.impl.client.DefaultRedirectStrategy:

RedirectLocations redirectLocations = (RedirectLocations) clientContext.getAttribute(
                HttpClientContext.REDIRECT_LOCATIONS);

The value of the clientContext should be basicHttpContext, but Spring Web (4.3.x.RELEASE) is initializing the context in:

org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal();

The value of clientContext is changing, which results in circular redirect error. The Spring Web (3.2.x.RELEASE) don't initialize the context and value will be null.

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.