5

I am trying to get more information about what is going on when I connect to a website at a lower level than what HttpWebRequest and HttpWebResponse gives me. I am using C#.

I would like to be able to see information about the dns lookup and the time it took to establish a connection (if a new connection was established). HttpWebRequest and HttpWebResponse work at a higher level than this and I want to ask if there is a way of getting the underlying TcpClient object (or whatever low level object they use).

If it is not possible, then is there a way to grab and manipulate a list of the connections that .net is maintaining without getting it through HttpWebRequest or HttpWebResponse?

I can't change the application I am working on to use TcpClient because it would be too time consuming to implement all the http stuff reliably.

2
  • 1
    Not sure about 'TcpClient' but I'm sure they use the 'Socket' class so look for it. Additionally, when you break-point and debug,you point over objects and see even their private members. That should give you some direction.
    – Poni
    Jan 3, 2011 at 16:12
  • I hadn't thought about looking at the private members, thank you. Unfortunately I couldn't find much. I did see that the ServicePoint has a non-public variable called LastDnsResolve which I assume holds the time of the last DNS resolution. I may be able to use that for a small piece of my problem.
    – Zane
    Jan 3, 2011 at 16:43

3 Answers 3

4

The best that I can get you is to create an app.config file with the following information:

<?xml version="1.0" encoding="UTF-8" ?>
<configuration>
    <system.diagnostics>
        <trace autoflush="true" />
            <sources>
                <source name="System.Net" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>
              <source name="System.Net.Sockets" maxdatasize="1024">
                    <listeners>
                        <add name="MyTraceFile"/>
                    </listeners>
                </source>  
           </sources>


            <sharedListeners>
                <add
                  name="MyTraceFile"
                  type="System.Diagnostics.TextWriterTraceListener"
                  initializeData="System.Net.trace.log"
                />
            </sharedListeners>
            <switches>
                <add name="System.Net" value="Verbose" />
              <add name="System.Net.Sockets" value="Verbose" />
            </switches>
    </system.diagnostics>
</configuration>

This will enable tracing and will kick out a log file named "System.Net.trace.log" in your app folder. You aren't going to get all of the information that you're looking for and its not easily consumable while the app is running but at least you don't need to have a third-party program running. Its not documented too much but there's some information out there at least.

5
  • 2
    Building on this idea, you could write your own TraceListener which would enable you to track all of the logs in the same process at runtime. Jan 3, 2011 at 17:59
  • Thank you, this seems close. I have written a TraceListener as suggested and can get some information from the socket traces. Unfortunately however I can't match a particular trace item, which shows socket information, to the particular HttpWebResponse that it is associated with.
    – Zane
    Jan 3, 2011 at 21:07
  • Frustratingly, I can see that .NET is using sockets behind the scenes, but can't access them. Is there a way to get an object from .NET using one of the unique object IDs that are displayed in the traces?
    – Zane
    Jan 3, 2011 at 21:09
  • You can use Trace.CorrelationManager.ActivityId. It's a per-thread value that is automatically included in traces. I don't think that HttpWebResponse uses it, so you'd have to set it yourself. Jan 4, 2011 at 12:39
  • I know this is a really late accept but I never managed to get this working as planned. I think this was the best suggestion though. I ran into some trouble trying to parse the messages in my TraceListener and couldn't find a way to reliably string all the information about a single request together because many threads were running at once.
    – Zane
    Oct 21, 2011 at 16:30
2

Use Wireshark, it's the best way to find out all that stuff.

3
  • I agree. The point of HttpWebRequest is to abstract/hide the complicated parts. The tradeoff for ease of use is that you have to accept some "magic" at the lower levels.
    – Chris Haas
    Jan 3, 2011 at 16:21
  • I agree. Instead of mucking about the internals of .net, just monitor the requests as they are going over the wire. This should give you exact timings as to when everything occurs.
    – NotMe
    Jan 3, 2011 at 16:21
  • I do use wireshark, but I would like to do the same thing in .net: monitor the tcp connections. My question may have been unclear, I would like to be able to get this information repeatedly and automatically in .net and not just for once off troubleshooting. Unless you are suggesting that I use WinPcap or a Wireshark API (does that exist?). But that is taking it to another level and I would like to avoid that if possible.
    – Zane
    Jan 3, 2011 at 16:36
1

If not Wireshark, then use Fiddler.

2
  • I use fiddler too. But that makes me think, how does fiddler get its statistical information? It reports on TCP/IP connection time and is written in C#. Does it use TcpClient/Sockets? Fiddler is very extensive in what it can do. I would feel too guilty to disassemble it, see how it's done and then use the same idea because I feel that's wrong (and it's probably illegal).
    – Zane
    Jan 3, 2011 at 16:49
  • 2
    Fiddler is HTTP proxy - so whe Fiddled is turned on browser/any other HTTP client sends requests to local proxy (Fiddler) and it resends them to actual server. As result it knows everything about requests and responses since it send them itself. But it does not sound like what you need from original question - you seem to need HTTP/TCP stats for your process... Jan 3, 2011 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.