104

I am using tcpdump to get HTTP data by executing the below command:

sudo tcpdump -A -s 1492 dst port 80

The result of above command:

  1. Headers, I think request and response headers.
  2. Unreadable data.
  3. The url GET /modules/mod_news_pro_gk1/cache/stories.ilbalad.ajayeb.strange-tractor.jpg.

I need a more clear result, for example, readable request > response header > response body etc. How can I filter my results?

3
  • 3
    HTTP responses may be compressed or chunked and getting their raw text may not really do what you need ("unreadable data"). You probably need to explain more completely what you want to do and what your environment permits (e.g. can't you just use a HTTP proxy like Charles or Fiddler)?
    – EricLaw
    Commented Jan 23, 2011 at 22:50
  • how did you run tcpdump on Android OS? Do you somehow embed the command in your app or you run it on a PC that is connected to the Android phone?
    – faizal
    Commented Jul 25, 2014 at 7:13
  • 1
    @faizal - you should install the tcpdump binary file into the device, then you can run it from the device itself, please use this link for How to install tcpdump into Android device : gadgetcat.wordpress.com/2011/09/11/tcpdump-on-android
    – kimo
    Commented Jul 25, 2014 at 21:33

4 Answers 4

244

There are tcpdump filters for HTTP GET & HTTP POST (or for both plus message body):

  • Run man tcpdump | less -Ip examples to see some examples

  • Here’s a tcpdump filter for HTTP GET (GET = 0x47, 0x45, 0x54, 0x20):

    sudo tcpdump -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'
    
  • Here’s a tcpdump filter for HTTP POST (POST = 0x50, 0x4f, 0x53, 0x54):

    sudo tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)'
    
  • Monitor HTTP traffic including request and response headers and message body (source):

    tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
    tcpdump -X -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
    

For more information on the bit-twiddling in the TCP header see: String-Matching Capture Filter Generator (link to Sake Blok's explanation).

8
  • 21
    I didn't know about less -Ip - what a timesaver
    – n611x007
    Commented Nov 7, 2013 at 12:36
  • 11
    I'd like to see an explanation of those tcpdump invokations
    – tmc
    Commented Jan 31, 2014 at 3:17
  • 1
    Is it possible to have a filter for HTTP GET and POST together?
    – Jim Ho
    Commented Feb 4, 2016 at 4:01
  • 4
    This is sorcery. Someone please explain how it works.
    – user3905644
    Commented Jul 14, 2017 at 7:40
  • 2
    explainshell.com/…
    – jmunsch
    Commented Dec 14, 2018 at 20:32
33

I would recommend using Wireshark, which has a "Follow TCP Stream" option that makes it very easy to see the full requests and responses for a particular TCP connection. If you would prefer to use the command line, you can try tcpflow, a tool dedicated to capturing and reconstructing the contents of TCP streams.

Other options would be using an HTTP debugging proxy, like Charles or Fiddler as EricLaw suggests. These have the advantage of having specific support for HTTP to make it easier to deal with various sorts of encodings, and other features like saving requests to replay them or editing requests.

You could also use a tool like Firebug (Firefox), Web Inspector (Safari, Chrome, and other WebKit-based browsers), or Opera Dragonfly, all of which provide some ability to view the request and response headers and bodies (though most of them don't allow you to see the exact byte stream, but instead how the browsers parsed the requests).

And finally, you can always construct requests by hand, using something like telnet, netcat, or socat to connect to port 80 and type the request in manually, or a tool like htty to help easily construct a request and inspect the response.

6
  • 2
    Thanks a lot for your answer Brain. I am using tcpdump in android OS :) therefore i am limited by using tcpdump,for sure I can get the headers and also the link request, i need also to get the body, any help, any Perl convert script or something like that?
    – kimo
    Commented Jan 24, 2011 at 11:32
  • 11
    @kimo In that case, I'd recommend using tcpdump -w outfile ... to dump the raw packets into a file, and then copy it to your computer, and use Wireshark to analyze it. Wireshark can open up a packet dump, decode the TCP stream, and decompress the HTTP content (I'm assuming that the reason it's unreadable is because it's compressed). Commented Jan 24, 2011 at 18:37
  • Thanks Brian, i will try to implement this solution, anyway it's better to move the file to PC, because i should filter the data to get the wanted requests, and it is not easy to do it on android system.
    – kimo
    Commented Jan 24, 2011 at 23:23
  • Brian I think what we are trying to say here is. How can we foo this with Busy Box. I am having the same constraints on Android devices also. Commented Nov 23, 2013 at 3:28
  • 1
    @JamesAndino Yes, on an Android device, I would recommend using tcpdump -w outfile, then copying that to a machine that has Wireshark and using Wireshark to view the dump. Commented Nov 23, 2013 at 6:14
7

Here is another choice: Chaosreader

So I need to debug an application which posts xml to a 3rd party application. I found a brilliant little perl script which does all the hard work – you just chuck it a tcpdump output file, and it does all the manipulation and outputs everything you need...

The script is called chaosreader0.94. See http://www.darknet.org.uk/2007/11/chaosreader-trace-tcpudp-sessions-from-tcpdump/

It worked like a treat, I did the following:

tcpdump host www.blah.com -s 9000 -w outputfile; perl chaosreader0.94 outputfile
3

This is what I tend to use:

tcpdump -s0 -Aql host example.com and port 80 -i any | grep -A5 -P 'HTTP'

Where -A5 is the number of lines you want to match after HTTP.

1
  • 1
    Since HTTP headers tend to end with a blank line, something like sed '/HTTP/,/^$/!d' might work instead of grep.
    – mwfearnley
    Commented Jun 13, 2023 at 16:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.