226

I am in a situation where when I get an HTTP 400 code from the server, it is a completely legal way of the server telling me what was wrong with my request (using a message in the HTTP response content)

However, the .NET HttpWebRequest raises an exception when the status code is 400.

How do I handle this? For me a 400 is completely legal, and rather helpful. The HTTP content has some important information but the exception throws me off my path.

1

7 Answers 7

395

It would be nice if there were some way of turning off "throw on non-success code" but if you catch WebException you can at least use the response:

using System;
using System.IO;
using System.Web;
using System.Net;

public class Test
{
    static void Main()
    {
        WebRequest request = WebRequest.Create("http://csharpindepth.com/asd");
        try
        {
            using (WebResponse response = request.GetResponse())
            {
                Console.WriteLine("Won't get here");
            }
        }
        catch (WebException e)
        {
            using (WebResponse response = e.Response)
            {
                HttpWebResponse httpResponse = (HttpWebResponse) response;
                Console.WriteLine("Error code: {0}", httpResponse.StatusCode);
                using (Stream data = response.GetResponseStream())
                using (var reader = new StreamReader(data))
                {
                    string text = reader.ReadToEnd();
                    Console.WriteLine(text);
                }
            }
        }
    }
}

You might like to encapsulate the "get me a response even if it's not a success code" bit in a separate method. (I'd suggest you still throw if there isn't a response, e.g. if you couldn't connect.)

If the error response may be large (which is unusual) you may want to tweak HttpWebRequest.DefaultMaximumErrorResponseLength to make sure you get the whole error.

15
  • 6
    The contents of the stream returned by GetResponseStream() on the response attached to the WebException is just the name of the status code (e.g. "Bad Request") rather than the response actually returned by the server. Is there any way to get this information?
    – Mark Watts
    Nov 7, 2012 at 9:52
  • @MarkWatts: It should be whatever's been returned by the server, and has been in every situation I've seen. Can you reproduce this with a particular external URL? I suggest you ask a new question (referring to this one) and showing what's going on.
    – Jon Skeet
    Nov 7, 2012 at 9:54
  • Turns out that it only does this when the content length of the response is zero; it adds a textual description of the HTTP status code - 400 is just "Bad Request" but some of the others are more descriptive.
    – Mark Watts
    Nov 7, 2012 at 13:01
  • If anybody knows of a nice wrapper for this class drop a link to it here. System.Net.WebClient behaves the same way by invoking the exception handling system.
    – John K
    May 27, 2013 at 19:00
  • 1
    @AnkushJain: I believe it will return normally for 2XX; redirection may happen for 3XX depending on configuration - I'd expect anything else to cause an exception, though I could be wrong. (For 4XX, it's just possible that it will then apply auth information if it has it but hadn't already used it.)
    – Jon Skeet
    Apr 5, 2017 at 10:31
52

I know this has already been answered a long time ago, but I made an extension method to hopefully help other people that come to this question.

Code:

public static class WebRequestExtensions
{
    public static WebResponse GetResponseWithoutException(this WebRequest request)
    {
        if (request == null)
        {
            throw new ArgumentNullException("request");
        }

        try
        {
            return request.GetResponse();
        }
        catch (WebException e)
        {
            if (e.Response == null)
            {
                throw;
            }

            return e.Response;
        }
    }
}

Usage:

var request = (HttpWebRequest)WebRequest.CreateHttp("http://invalidurl.com");

//... (initialize more fields)

using (var response = (HttpWebResponse)request.GetResponseWithoutException())
{
    Console.WriteLine("I got Http Status Code: {0}", response.StatusCode);
}
5
  • 2
    WebException.Response can and may be null. You should rethrow if this is the case.
    – Ian Kemp
    Apr 25, 2016 at 11:15
  • @DavidP I agree, the usage is a little janky, though for most things, you should probably be using HttpClient instead, it's much more configurable, and I believe it's the way of the future.
    – Matthew
    Aug 29, 2017 at 13:27
  • Is checking for request == null really necessary? Since this is an extension method, trying to use it on a null object should throw a null reference exception before it hits the extension method code......right?
    – kwill
    May 15, 2020 at 16:04
  • @kwill Extension methods are just static methods, doing proper input validation should be done to avoid confusion, especially when they're not invoked with the extension method syntax. Also, this is the consistent approach done by the the .NET teams: github.com/dotnet/corefx/blob/…
    – Matthew
    May 15, 2020 at 20:44
  • 1
    Sorry I misunderstood your second question. ((WebRequest) null).GetResponseWithoutException() will in fact not cause a NullReferenceException, since it is compiled to the equivalent of WebRequestExtensions.GetResponseWithoutException(null), which would not result in a NullReferenceException, hence the need for input validation.
    – Matthew
    May 15, 2020 at 20:47
14

Interestingly, the HttpWebResponse.GetResponseStream() that you get from the WebException.Response is not the same as the response stream that you would have received from server. In our environment, we're losing actual server responses when a 400 HTTP status code is returned back to the client using the HttpWebRequest/HttpWebResponse objects. From what we've seen, the response stream associated with the WebException's HttpWebResponse is generated at the client and does not include any of the response body from the server. Very frustrating, as we want to message back to the client the reason for the bad request.

1
  • I use method HEAD and cause this exception but when use GET there is no any problem. What is the issue with HEAD method exactly? May 25, 2018 at 20:33
13

I had similar issues when trying to connect to Google's OAuth2 service.

I ended up writing the POST manually, not using WebRequest, like this:

TcpClient client = new TcpClient("accounts.google.com", 443);
Stream netStream = client.GetStream();
SslStream sslStream = new SslStream(netStream);
sslStream.AuthenticateAsClient("accounts.google.com");

{
    byte[] contentAsBytes = Encoding.ASCII.GetBytes(content.ToString());

    StringBuilder msg = new StringBuilder();
    msg.AppendLine("POST /o/oauth2/token HTTP/1.1");
    msg.AppendLine("Host: accounts.google.com");
    msg.AppendLine("Content-Type: application/x-www-form-urlencoded");
    msg.AppendLine("Content-Length: " + contentAsBytes.Length.ToString());
    msg.AppendLine("");
    Debug.WriteLine("Request");
    Debug.WriteLine(msg.ToString());
    Debug.WriteLine(content.ToString());

    byte[] headerAsBytes = Encoding.ASCII.GetBytes(msg.ToString());
    sslStream.Write(headerAsBytes);
    sslStream.Write(contentAsBytes);
}

Debug.WriteLine("Response");

StreamReader reader = new StreamReader(sslStream);
while (true)
{  // Print the response line by line to the debug stream for inspection.
    string line = reader.ReadLine();
    if (line == null) break;
    Debug.WriteLine(line);
}

The response that gets written to the response stream contains the specific error text that you're after.

In particular, my problem was that I was putting endlines between url-encoded data pieces. When I took them out, everything worked. You might be able to use a similar technique to connect to your service and read the actual response error text.

2
  • 2
    HttpWebRequest is so messed up. Even sockets are easier (coz they don't hide errors away from you).
    – Agent_L
    Feb 26, 2012 at 11:04
  • So how do I set the body?
    – ilidiocn
    Aug 30, 2021 at 12:21
6

Try this (it's VB-Code :-):

Try

Catch exp As WebException
  Dim sResponse As String = New StreamReader(exp.Response.GetResponseStream()).ReadToEnd
End Try
0
4

An asynchronous version of extension function:

    public static async Task<WebResponse> GetResponseAsyncNoEx(this WebRequest request)
    {
        try
        {
            return await request.GetResponseAsync();
        }
        catch(WebException ex)
        {
            return ex.Response;
        }
    }
-1

This solved it for me:
https://gist.github.com/beccasaurus/929007/a8f820b153a1cfdee3d06a9c0a1d7ebfced8bb77

TL;DR:
Problem:
localhost returns expected content, remote IP alters 400 content to "Bad Request"
Solution:
Adding <httpErrors existingResponse="PassThrough"></httpErrors> to web.config/configuration/system.webServer solved this for me; now all servers (local & remote) return the exact same content (generated by me) regardless of the IP address and/or HTTP code I return.

3
  • This has nothing to do with WebHttpRequest throwing exceptions on unsuccessful HTTP codes.
    – GSerg
    Feb 16, 2021 at 22:02
  • @GSerg then what is the cause? Don't just criticize without including some insight as to why you believe your comment is valid.
    – dlchambers
    Feb 18, 2021 at 13:09
  • The cause is that the WebHttpRequest class is written to throw exceptions on unsuccessful HTTP codes, which many people find unhelpful. It has nothing to do with web.config configuration which controls the server settings. The issue you are talking about (the observed HTTP payload) has nothing to do with the fact that WebHttpRequest throws exceptions on unsuccessful HTTP codes.
    – GSerg
    Feb 18, 2021 at 14:18

Not the answer you're looking for? Browse other questions tagged or ask your own question.