38

In this answer, I described how I resorted to wrappnig a GZipStream around the response stream in a HttpWebResponse, in order to decompress it.

The relevant code looks like this:

HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
hwr.CookieContainer =
    PersistentCookies.GetCookieContainerForUrl(url);
hwr.Accept = "text/xml, */*";
hwr.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
hwr.Headers.Add(HttpRequestHeader.AcceptLanguage, "en-us");
hwr.UserAgent = "My special app";
hwr.KeepAlive = true;

using (var resp = (HttpWebResponse) hwr.GetResponse()) 
{
    using(Stream s = resp.GetResponseStream())
    {
        Stream s2 = s;
        if (resp.ContentEncoding.ToLower().Contains("gzip"))
            s2 = new GZipStream(s2, CompressionMode.Decompress);
        else if (resp.ContentEncoding.ToLower().Contains("deflate"))
            s2 = new DeflateStream(s2, CompressionMode.Decompress);

         ... use s2 ...
    }
}

Is there a way to get HttpWebResponse to provide a de-compressing stream, automatically? In other words, a way for me to eliminate the following from the above code:

      Stream s2 = s;
      if (resp.ContentEncoding.ToLower().Contains("gzip"))
          s2 = new GZipStream(s2, CompressionMode.Decompress);
      else if (resp.ContentEncoding.ToLower().Contains("deflate"))
          s2 = new DeflateStream(s2, CompressionMode.Decompress);

Thanks.

3 Answers 3

74

Use the HttpWebRequest.AutomaticDecompression property as follows:

HttpWebRequest hwr = (HttpWebRequest) WebRequest.Create(url);
hwr.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;

It's not necessary to manually add the Accept-Encoding HTTP header; it will automatically be added when that property is used.

(Also, I know this is just example code, but the HttpWebResponse object should be placed in a using block so it's disposed correctly when you've finished using it.)

2
  • Cha-ching! Thanks. How'd I miss that?
    – Cheeso
    Commented May 12, 2010 at 3:03
  • 5
    "It's not necessary to manually add the Accept-Encoding HTTP header" - Furthermore, if you do, the "gzip" and "deflate" values will be duplicated in the header which may cause some servers to reject the request.
    – Chad
    Commented Aug 5, 2013 at 0:29
0

My expreience is that it already does it automatically. I had to explicitly disable it by setting the AutomaticDecompression property of the Request object to DecompressionMethods.None

4
  • 1
    Hmm, that's odd. I need to re-run my tests. I thought I saw compressed data coming through. I may be wrong. thanks.
    – Cheeso
    Commented May 12, 2010 at 3:00
  • 1
    Just tested - in my code it was not automatically on. I'll have to read the doc to find out, but. . . when I explicitly enabled it, it eliminated the need for me to do the decompression stream thing. thanks for the answer.
    – Cheeso
    Commented May 12, 2010 at 3:04
  • I had a a hard time actually finding a site that would compress my data as requested -- google.com wouldn't do it. -- On a musing I put in the URL for this exact StackOverflow question, and it worked -- when I leave it set to DecompressionMethods.None (the apparent default in .NET 4.5.1), I get gzipped bytes back. Nice. Commented Jan 11, 2014 at 23:45
  • Depending on the version of the framework you have the default is different Commented Jun 21, 2018 at 11:39
-1

I dont think AutomaticDecompression adds to Header AcceptDecoding=gzip,deflate. Below is example of my request and response when I added just AutomaticDecompression (without adding explicityly AcceptEncoding header). As you can see, there is no AcceptEncoding in Header at all

//REQUEST 
//===============================================================
-       req {System.Net.HttpWebRequest} System.Net.HttpWebRequest
+       base    {System.Net.HttpWebRequest} System.Net.WebRequest {System.Net.HttpWebRequest}
        _AuthInfo   null    System.Net.ICredentials
        _ContentLength  -1  long
+       _HttpRequestHeaders {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
Authorization: Basic RElOT0I6MTExMQ==

}   System.Net.WebHeaderCollection
        _MaximumAllowedRedirections 50  int
        _MediaType  null    string
+       _OriginUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        _OriginVerb "GET"   string
+       _Proxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
        _ProxyAuthenticationState   null    System.Net.AuthenticationState
        _ReadWriteTimeout   300000  int
        _ServerAuthenticationState  null    System.Net.AuthenticationState
        _Timeout    100000  int
+       _Timer  {System.Threading.Timer}    System.Threading.Timer
+       _Uri    {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        Aborted false   bool
        Accept  Could not evaluate expression   string
+       Address {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        AllowAutoRedirect   false   bool
        AllowWriteStreamBuffering   true    bool
        AutomaticDecompression  GZip | Deflate  System.Net.DecompressionMethods
        ChallengedUri   Could not evaluate expression   System.Uri
+       ClientCertificates  {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
        ConnectHostAndPort  "192.168.0.106:8084"    string
        Connection  Could not evaluate expression   string
        ConnectionGroupName Could not evaluate expression   string
        ContentLength   -1  long
        ContentType Could not evaluate expression   string
        ContinueDelegate    Could not evaluate expression   System.Net.HttpContinueDelegate
        Credentials Could not evaluate expression   System.Net.ICredentials
+       CurrentAuthenticationState  {System.Net.AuthenticationState}    System.Net.AuthenticationState
        Expect  Could not evaluate expression   string
        hasEntityData   false   bool
        HaveResponse    false   bool
+       Headers {User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)
Authorization: Basic RElOT0I6MTExMQ==

}   System.Net.WebHeaderCollection
+       IfModifiedSince {9/7/15 5:47:12 AM} System.DateTime
        KeepAlive   true    bool
        m_Aborted   false   bool
        m_allowAutoRedirection  false   bool
        m_allowWriteStreamBuffering true    bool
        m_AutomaticDecompression    GZip | Deflate  System.Net.DecompressionMethods
        m_boundConnection   null    System.Net.Connection
+       m_clientCertificates    {System.Security.Cryptography.X509Certificates.X509CertificateCollection}   System.Security.Cryptography.X509Certificates.X509CertificateCollection
        m_connection    null    System.Net.Connection
        m_ConnectionClosedStopSendingEntityData false   bool
        m_connectionGroupName   null    string
        m_connectionUsers   0   int
        m_connMgrConnection -1  int
        m_continueDelegate  null    System.Net.HttpContinueDelegate
+       m_continueFunction  {System.Threading.TimerCallback}    System.Threading.TimerCallback
+       m_continueTimer null    System.Threading.Timer
        m_doneSendingEvent  null    System.Threading.ManualResetEvent
        m_errorResponse null    System.Net.WebException
        m_expectContinue    false   bool
        m_finishedWriting   false   bool
        m_httpWriteMode None    System.Net.HttpWriteMode
        m_IsCurrentAuthenticationStateProxy false   bool
        m_isSubmitting  false   bool
        m_KeepAlive true    bool
        m_numberRedirections    0   int
        m_Pipelined true    bool
        m_PreAuthenticate   false   bool
+       m_readWriteTimer    {System.Threading.Timer}    System.Threading.Timer
        m_requestGeneration 0   int
+       m_requestSentEvent  {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
+       m_requestStream null    System.Net.HttpWriteStream
        m_requestStreamRetrieved    false   bool
        m_RequestSubmitted  false   bool
        m_response  null    System.Net.HttpWebResponse
        m_responseComplete  false   bool
+       m_responseEvent {System.Threading.ManualResetEvent} System.Threading.ManualResetEvent
        m_responseRetrieved false   bool
        m_sentHeaders   false   bool
+       m_setProxy  {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
+       m_srvPoint  null    System.Net.ServicePoint
        m_startedReceiving  false   bool
        m_timedOut  0   int
+       m_version   {1.1}   System.Version
        MaximumAutomaticRedirections    50  int
        MediaType   Could not evaluate expression   string
        Method  "GET"   string
        NtlmKeepAlive   false   bool
        Pipelined   true    bool
        PreAuthenticate false   bool
+       ProtocolVersion {1.1}   System.Version
+       Proxy   {System.Net.GlobalProxySelection.SystemWebProxy}    System.Net.IWebProxy {System.Net.GlobalProxySelection.SystemWebProxy}
        ProxyAuthenticatedConnectionSharing false   bool
+       ProxyAuthenticationState    {System.Net.AuthenticationState}    System.Net.AuthenticationState
        ReadWriteTimeout    300000  int
        Referer Could not evaluate expression   string
+       RequestUri  {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
+       ResponseStatusCode  'req.ResponseStatusCode' threw an exception of type 'System.NullReferenceException' System.Net.HttpStatusCode {System.NullReferenceException}
        SendChunked false   bool
+       ServerAuthenticationState   {System.Net.AuthenticationState}    System.Net.AuthenticationState
        ServicePoint    Could not evaluate expression   System.Net.ServicePoint
        SyncObject  {object}    object
        Timeout 100000  int
        TransferEncoding    Could not evaluate expression   string
        UserAgent   "Mozilla/4.0 (compatible; MSIE 6.0; MS .NET CF Web Services Client Protocol 3.5.7283.0)"    string
+       UsesProxySemantics  'req.UsesProxySemantics' threw an exception of type 'System.NullReferenceException' bool {System.NullReferenceException}
+       Static members  


//RESPONSE 
=====================================================
-       res {System.Net.HttpWebResponse}    System.Net.HttpWebResponse
+       base    {System.Net.HttpWebResponse}    System.Net.WebResponse {System.Net.HttpWebResponse}
        CharacterSet    ""  string
        ContentEncoding ""  string
        ContentLength   -1  long
        ContentType "text/xml"  string
-       Headers {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Net.WebHeaderCollection
+       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
        m_IsHttpWebHeaderObject true    bool
+       Static members      
+       LastModified    {9/7/15 5:49:45 AM} System.DateTime
        m_contentLength -1  long
        m_decompressionMethod   GZip | Deflate  System.Net.DecompressionMethods
-       m_httpResponseHeaders   {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Net.WebHeaderCollection
-       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameValueCollection {System.Net.WebHeaderCollection}
+       base    {Date: Mon, 07 Sep 2015 11:49:36 GMT
Server: Apache/2.2.29 (Win32)
Vary: Accept-Encoding,User-Agent
Content-Encoding: 
Content-Length: 1481
Keep-Alive: timeout=60, max=100
Connection: Keep-Alive
Content-Type: text/xml

}   System.Collections.Specialized.NameObjectCollectionBase {System.Net.WebHeaderCollection}
        _allKeys    null    string[]
+       AllKeys {string[8]} string[]
        m_IsHttpWebHeaderObject true    bool
+       Static members      
        m_mediaType null    string
        m_method    "POST"  string
-       m_responseStream    {System.Net.GZipWrapperStream}  System.IO.Stream {System.Net.GZipWrapperStream}
+       [System.Net.GZipWrapperStream]  {System.Net.GZipWrapperStream}  System.Net.GZipWrapperStream
        base    {System.Net.GZipWrapperStream}  System.MarshalByRefObject {System.Net.GZipWrapperStream}
        _closeRecursionCounter  0   int
        CanRead true    bool
        CanSeek false   bool
        CanTimeout  false   bool
        CanWrite    false   bool
+       Length  'res.m_responseStream.Length' threw an exception of type 'System.NotSupportedException' long {System.NotSupportedException}
+       Position    'res.m_responseStream.Position' threw an exception of type 'System.NotSupportedException'   long {System.NotSupportedException}
+       ReadTimeout 'res.m_responseStream.ReadTimeout' threw an exception of type 'System.InvalidOperationException'    int {System.InvalidOperationException}
+       WriteTimeout    'res.m_responseStream.WriteTimeout' threw an exception of type 'System.InvalidOperationException'   int {System.InvalidOperationException}
+       Static members      
        m_statusCode    200 int
        m_statusDescription "OK"    string
+       m_url   {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
+       m_version   {1.1}   System.Version
        Method  "POST"  string
-       ProtocolVersion {1.1}   System.Version
        _Build  -1  int
        _Major  1   int
        _Minor  1   int
        _Revision   -1  int
        Build   -1  int
        Major   1   int
        Minor   1   int
        Revision    -1  int
-       ResponseUri {http://192.168.0.106:8084/MyTestApp/Soap/IMyApp}   System.Uri
        AbsolutePath    "/MyTestApp/Soap/IMyApp"    string
        AbsoluteUri "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
        Authority   "192.168.0.106:8084"    string
        DnsSafeHost "192.168.0.106" string
        Fragment    ""  string
        HasAuthority    true    bool
        Host    "192.168.0.106" string
        HostNameType    IPv4    System.UriHostNameType
        HostType    IPv4HostType    System.Uri.Flags
        IsAbsoluteUri   true    bool
        IsDefaultPort   false   bool
        IsDosPath   false   bool
        IsFile  false   bool
        IsImplicitFile  false   bool
        IsLoopback  false   bool
        IsNotAbsoluteUri    false   bool
        IsUnc   false   bool
        IsUncOrDosPath  false   bool
        IsUncPath   false   bool
        LocalPath   "/MyTestApp/Soap/IMyApp"    string
        m_Flags IPv4HostType | AuthorityFound | NotDefaultPort | MinimalUriInfoSet | AllUriInfoSet  System.Uri.Flags
+       m_Info  {System.Uri.UriInfo}    System.Uri.UriInfo
        m_OrigFileString    null    string
        m_String    "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
+       m_Syntax    {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
        OriginalString  "http://192.168.0.106:8084/MyTestApp/Soap/IMyApp"   string
        PathAndQuery    "/MyTestApp/Soap/IMyApp"    string
        Port    8084    int
        PrivateAbsolutePath "/MyTestApp/Soap/IMyApp"    string
        Query   ""  string
        Scheme  "http"  string
        SecuredPathIndex    0   ushort
+       Segments    {string[4]} string[]
+       Syntax  {System.UriParser.BuiltInUriParser} System.UriParser {System.UriParser.BuiltInUriParser}
        UserDrivenParsing   false   bool
        UserEscaped false   bool
        UserInfo    ""  string
+       Static members      
        Server  "Apache/2.2.29 (Win32)" string
        StatusCode  OK  System.Net.HttpStatusCode
        StatusDescription   "OK"    string
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.