EXAMPLES.md (17329B)
1 Some helpful tips: 2 3 * When the intention is to match all packets (i.e. blank regex), it is 4 technically faster to use an empty regex (`''`) than to use a wildcard 5 (e.g. `'.*'`, `'*'`). 6 7 * When sniffing interfaces that are very busy or are seeing large amounts of 8 packet traffic, make sure to craft a BPF filter to limit what PCAP has to 9 deliver to ngrep. The ngrep parser takes a certain amount of time and while 10 negligible on a slow interface, it can add up very quickly on a busy one. 11 12 * Hexadecimal expressions can be in straight numeric form, 'DEADBEEF', or in 13 symbolic form, '0xDEADBEEF'. A byte is the smallest unit of measure you can 14 match against. 15 16 * As of v1.28, ngrep doesn't require a match expression. However, there are 17 cases where ngrep can be confused and think part of your bpf filter is the 18 match expression, as in: 19 20 ``` 21 % ngrep not port 80 22 interface: eth0 (192.168.1.0/255.255.255.0) 23 filter: ip and ( port 80 ) 24 match: not 25 ``` 26 27 In cases like this, you will need to specify a blank match expression: 28 29 ``` 30 % ngrep '' not port 80 31 interface: eth0 (192.168.1.0/255.255.255.0) 32 filter: ip and ( not port 80 ) 33 ``` 34 35 ## Basic Packet Sniffing 36 37 Basic packet sniffing is easy with ngrep. It supports BPF filter logic, which 38 means to say constraining what ngrep sees and displays is as easy as saying 39 something like `ngrep host foo.bar.com and port 25`. Following are a few 40 examples of common invocations of ngrep to do basic packet sniffing. Please 41 note the usage of `any' as the specified ethernet adaptor to attach to; in most 42 recent UNIX libpcap implementations this will instruct ngrep to attach to all 43 interfaces at once, local (lo) and all external interfaces that may be active. 44 45 * `ngrep -d any port 25` 46 47 Monitor all activity crossing source or destination port 25 (SMTP). 48 49 * `ngrep -d any 'error' port syslog` 50 51 Monitor any network-based syslog traffic for the occurrence of the word "error". 52 ngrep knows how to convert service port names (on UNIX, located in 53 `/etc/services`) to port numbers. 54 55 * `ngrep -wi -d any 'user|pass' port 21` 56 57 Monitor any traffic crossing source or destination port 21 (FTP), looking 58 case-insensitively for the words "user" or "pass", matched as word-expressions 59 (the match term(s) must have non-alphanumeric, delimiting characters surrounding 60 them). 61 62 ## Debugging HTTP interactions 63 64 In certain scenarios it is desirous to see how web browsers communicate with web 65 servers, and to inspect the HTTP headers and possibly cookie values that they 66 are exchanging. 67 68 In this example, we run an ngrep on a webserver. Since it only has 69 one interface, eth0, we omit specifying the interface manually on the 70 command line and allow ngrep to choose the default interface for us, 71 for convenience. 72 73 ``` 74 # ngrep port 80 75 interface: eth0 (64.90.164.72/255.255.255.252) 76 filter: ip and ( port 80 ) 77 #### 78 T 67.169.59.38:42167 -> 64.90.164.74:80 [AP] 79 GET / HTTP/1.1..User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i 80 686) Opera 7.21 [en]..Host: www.darkridge.com..Accept: text/html, applicat 81 ion/xml;q=0.9, application/xhtml+xml;q=0.9, image/png, image/jpeg, image/gi 82 f, image/x-xbitmap, */*;q=0.1..Accept-Charset: iso-8859-1, utf-8, utf-16, * 83 ;q=0.1..Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0..Cookie: SQ 84 MSESSID=5272f9ae21c07eca4dfd75f9a3cda22e..Cookie2: $Version=1..Connection: 85 Keep-Alive, TE..TE: deflate, gzip, chunked, identity, trailers.... 86 ## 87 T 64.90.164.74:80 -> 67.169.59.38:42167 [AP] 88 HTTP/1.1 200 OK..Date: Mon, 29 Mar 2004 00:44:40 GMT..Server: Apache/2.0.49 89 (Unix)..Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT..ETag: "210e23-326-f8 90 200b40"..Accept-Ranges: bytes..Vary: Accept-Encoding,User-Agent..Content-En 91 coding: gzip..Content-Length: 476..Keep-Alive: timeout=15, max=100..Connect 92 ion: Keep-Alive..Content-Type: text/html; charset=ISO-8859-1..Content-Langu 93 age: en..............}S]..0.|...........H...8........@..\....(.....Dw.%.,.. 94 ;.k.....Y>q<........d ...........3.i..kdm.u@d{.Q..\....@..B1.0.2YI^..R..... 95 ....X......X..y...\.....,..(........1...g.......*...j..a.`._@.W....0.....?. 96 .R.K.j..Y.....>...;kw*U.j.<...\0Tn.l.:......>Fs....'....h.'...u.H4..'.6.vID 97 I.......N.r.O...}...I.w. ...mX...L.s..{.L.R..-...e....~nu..t.3...H..#..J... 98 .u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m}..>/?..#........I 99 ..I..4.P......2:...n8l.......!.Yr&... 100 ## 101 ``` 102 103 As you can see, all headers and aspects of the HTTP transmission are exposed in 104 their gory detail. It's a little hard to parse though, so let's see what 105 happens when `-W byline` mode is used: 106 107 ``` 108 # ngrep -W byline port 80 109 interface: eth0 (64.90.164.72/255.255.255.252) 110 filter: ip and ( port 80 ) 111 #### 112 T 67.169.59.38:42177 -> 64.90.164.74:80 [AP] 113 GET / HTTP/1.1. 114 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; X11; Linux i686) Opera ... 115 Host: www.darkridge.com. 116 Accept: text/html, application/xml;q=0.9, application/xhtml+xml;q=0.9 ... 117 Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1. 118 Accept-Encoding: deflate, gzip, x-gzip, identity, *;q=0. 119 Cookie: SQMSESSID=5272f9ae21c07eca4dfd75f9a3cda22e. 120 Cookie2: $Version=1. 121 Cache-Control: no-cache. 122 Connection: Keep-Alive, TE. 123 TE: deflate, gzip, chunked, identity, trailers. 124 . 125 126 ## 127 T 64.90.164.74:80 -> 67.169.59.38:42177 [AP] 128 HTTP/1.1 200 OK. 129 Date: Mon, 29 Mar 2004 00:47:25 GMT. 130 Server: Apache/2.0.49 (Unix). 131 Last-Modified: Tue, 04 Nov 2003 12:09:41 GMT. 132 ETag: "210e23-326-f8200b40". 133 Accept-Ranges: bytes. 134 Vary: Accept-Encoding,User-Agent. 135 Content-Encoding: gzip. 136 Content-Length: 476. 137 Keep-Alive: timeout=15, max=100. 138 Connection: Keep-Alive. 139 Content-Type: text/html; charset=ISO-8859-1. 140 Content-Language: en. 141 . 142 ..........}S]..0.|...........H...8........@..\....(.....Dw.%.,..;.k.. ... 143 .;kw*U.j.<...\0Tn.l.:......>Fs....'....h.'...u.H4..'.6.vIDI.......N.r ... 144 ..H..#..J....u.?..]....^..2.....e8v/gP.....].48...qD!..........#y...m ... 145 #### 146 ``` 147 148 (Content visually truncated for display purposes.) 149 150 `-W byline` mode tells ngrep to respect embedded line feeds when they occur. 151 You'll note from the output above that there is still a trailing dot (".") on 152 each line, which is the carriage-return portion of the CRLF pair. Using this 153 mode, now the output has become much easier to visually parse. 154 155 ## Processing PCAP dump files, looking for patterns 156 157 I had a friend who worked at Network Solutions and among the things he did was 158 analyze huge 500M+ PCAP dump files of DNS traffic, looking for patterns and 159 anomalies. ngrep was an invaluable tool for this purpose; it allowed him to 160 take one instance of a network dump and search it quickly and repeatedly for 161 patterns in the data packets. 162 163 To save a PCAP dump file from ngrep is very easy; simply run ngrep as you 164 normally would but add one more command line option: `-O some.file.dump` (the 165 name of the file is largely irrelevant). To illustrate another feature of 166 ngrep, we will use the `-T` option (print time differential information). 167 168 ``` 169 # ngrep -O /tmp/dns.dump -d any -T port domain 170 interface: any 171 filter: ip and ( port domain ) 172 output: /tmp/dns.dump 173 # 174 U +0.000000 203.115.225.24:53 -> 64.90.164.74:53 175 .............m.razor2.cloudmark.com.......)........ 176 # 177 U +0.000281 64.90.164.74:53 -> 203.115.225.24:53 178 .............m.razor2.cloudmark.com................'.ns1...hostmaster..ws.. 179 ..p.... ..:.......)........ 180 # 181 U +0.078184 195.113.155.7:2949 -> 64.90.164.74:53 182 .............a.razor2.cloudmark.com..... 183 # 184 U +0.000351 64.90.164.74:53 -> 195.113.155.7:2949 185 .............a.razor2.cloudmark.com..................agony...4..........B.. 186 ..............ns1...............ns2...............ns3...X..........@Z.J.j.. 187 ........@Z...|..........B..; 188 ^Cexit 189 6 received, 0 dropped 190 ``` 191 192 Note the `output:` indicator and timestamp information. Now we have a PCAP dump 193 file, and so let's search it for some patterns: 194 195 ``` 196 # ngrep -w 'm' -I /tmp/dns.dump 197 input: /tmp/dns.dump 198 match: ((^m\W)|(\Wm$)|(\Wm\W)) 199 # 200 U 203.115.225.24:53 -> 64.90.164.74:53 201 .............m.razor2.cloudmark.com.......)........ 202 # 203 U 64.90.164.74:53 -> 203.115.225.24:53 204 .............m.razor2.cloudmark.com................'.ns1...hostmaster..ws.. 205 ..p.... ..:.......)........ 206 ##exit 207 ``` 208 209 Above we searched for the letter "m", matched as a word (`-w`). This yields two 210 packets. 211 212 ``` 213 # ngrep -tD ns3 -I /tmp/dns.dump 214 input: /tmp/dns.dump 215 match: ns3 216 #### 217 U 2004/03/28 20:32:37.088525 64.90.164.74:53 -> 195.113.155.7:2949 218 .............a.razor2.cloudmark.com..................agony...4..........B.. 219 ..............ns1...............ns2...............ns3...X..........@Z.J.j.. 220 ........@Z...|..........B..; 221 exit 222 ``` 223 224 Here we've added `-t` which means print the absolute timestamp on the packet, 225 and `-D` which means replay the packets by the time interval at which they were 226 recorded. The latter is a neat little feature for observing the traffic at the 227 rates/times they originally seen, though in this example it's not terribly 228 effective as there is only one packet being matched. 229 230 ``` 231 # ngrep -I /tmp/dns.dump port 80 232 input: /tmp/dns.dump 233 filter: ip and ( port 80 ) 234 exit 235 ``` 236 237 There's no port 80 traffic in the dump, so of course the BPF filter yields us no 238 results. 239 240 ## Observing binary being transferred across the wire 241 242 One interesting feature of ngrep is its ability to take a hexademical (binary) 243 expression and search for that in lieu of a regular expression. ngrep can also 244 display the packets it observes in a hexadecimal format, which is more effective 245 for inspecting binary content patterns. 246 247 In this example, we will simply look for a binary pattern in a web stream, but 248 the more obvious usage is to look for a DDoS Zombie's unique binary signature 249 (say, from a command packet), or even a Worm/Virus being transferred across the 250 wire as it propogates itself. 251 252 For this test, let's assume we have a GIF on a web server that has the data 253 pattern "0xc5d5e5f55666768696a6b6c6d6e6" (hexademical) in it. Once `-X` is 254 specified, the expression will be interpreted as a hexademical pattern instead 255 of a regular expression, and the "0x" prefix is optional. 256 257 To see a packet like this cross the wire: 258 259 ``` 260 # ngrep -xX '0xc5d5e5f55666768696a6b6c6d6e6' port 80 261 interface: eth0 (64.90.164.72/255.255.255.252) 262 filter: ip and ( port 80 ) 263 match: 0xc5d5e5f55666768696a6b6c6d6e6 264 ### 265 T 64.90.164.74:80 -> 67.169.59.38:42306 [A] 266 ff d8 ff e0 00 10 4a 46 49 46 00 01 02 01 00 48 ......JFIF.....H 267 00 48 00 00 ff ed 13 ba 50 68 6f 74 6f 73 68 6f .H......Photosho 268 70 20 33 2e 30 00 38 42 49 4d 03 ed 00 00 00 00 p 3.0.8BIM...... 269 00 10 00 48 00 00 00 01 00 01 00 48 00 00 00 01 ...H.......H.... 270 00 01 38 42 49 4d 04 0d 00 00 00 00 00 04 00 00 ..8BIM.......... 271 00 78 38 42 49 4d 03 f3 00 00 00 00 00 08 00 00 .x8BIM.......... 272 00 00 00 00 00 00 38 42 49 4d 04 0a 00 00 00 00 ......8BIM...... 273 00 01 00 00 38 42 49 4d 27 10 00 00 00 00 00 0a ....8BIM'....... 274 00 01 00 00 00 00 00 00 00 02 38 42 49 4d 03 f5 ..........8BIM.. 275 00 00 00 00 00 48 00 2f 66 66 00 01 00 6c 66 66 .....H./ff...lff 276 00 06 00 00 00 00 00 01 00 2f 66 66 00 01 00 a1 ........./ff.... 277 99 9a 00 06 00 00 00 00 00 01 00 32 00 00 00 01 ...........2.... 278 00 5a 00 00 00 06 00 00 00 00 00 01 00 35 00 00 .Z...........5.. 279 00 01 00 2d 00 00 00 06 00 00 00 00 00 01 38 42 ...-..........8B 280 49 4d 03 f8 00 00 00 00 00 70 00 00 ff ff ff ff IM.......p...... 281 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ................ 282 ff ff 03 e8 00 00 00 00 ff ff ff ff ff ff ff ff ................ 283 ff ff ff ff ff ff ff ff ff ff ff ff ff ff 03 e8 ................ 284 00 00 00 00 ff ff ff ff ff ff ff ff ff ff ff ff ................ 285 ff ff ff ff ff ff ff ff ff ff 03 e8 00 00 00 00 ................ 286 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ................ 287 ff ff ff ff ff ff 03 e8 00 00 38 42 49 4d 04 08 ..........8BIM.. 288 00 00 00 00 00 10 00 00 00 01 00 00 02 40 00 00 .............@.. 289 02 40 00 00 00 00 38 42 49 4d 04 14 00 00 00 00 .@....8BIM...... 290 00 04 00 00 00 06 38 42 49 4d 04 0c 00 00 00 00 ......8BIM...... 291 12 2a 00 00 00 01 00 00 00 70 00 00 00 57 00 00 .*.......p...W.. 292 01 50 00 00 72 30 00 00 12 0e 00 18 00 01 ff d8 .P..r0.......... 293 ff e0 00 10 4a 46 49 46 00 01 02 01 00 48 00 48 ....JFIF.....H.H 294 00 00 ff fe 00 26 46 69 6c 65 20 77 72 69 74 74 .....&File writt 295 65 6e 20 62 79 20 41 64 6f 62 65 20 50 68 6f 74 en by Adobe Phot 296 6f 73 68 6f 70 a8 20 35 2e 30 ff ee 00 0e 41 64 oshop. 5.0....Ad 297 6f 62 65 00 64 80 00 00 00 01 ff db 00 84 00 0c obe.d........... 298 08 08 08 09 08 0c 09 09 0c 11 0b 0a 0b 11 15 0f ................ 299 0c 0c 0f 15 18 13 13 15 13 13 18 11 0c 0c 0c 0c ................ 300 0c 0c 11 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c ................ 301 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 01 ................ 302 0d 0b 0b 0d 0e 0d 10 0e 0e 10 14 0e 0e 0e 14 14 ................ 303 0e 0e 0e 0e 14 11 0c 0c 0c 0c 0c 11 11 0c 0c 0c ................ 304 0c 0c 0c 11 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c ................ 305 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c 0c ................ 306 ff c0 00 11 08 00 57 00 70 03 01 22 00 02 11 01 ......W.p..".... 307 03 11 01 ff dd 00 04 00 07 ff c4 01 3f 00 00 01 ............?... 308 05 01 01 01 01 01 01 00 00 00 00 00 00 00 03 00 ................ 309 01 02 04 05 06 07 08 09 0a 0b 01 00 01 05 01 01 ................ 310 01 01 01 01 00 00 00 00 00 00 00 01 00 02 03 04 ................ 311 05 06 07 08 09 0a 0b 10 00 01 04 01 03 02 04 02 ................ 312 05 07 06 08 05 03 0c 33 01 00 02 11 03 04 21 12 .......3......!. 313 31 05 41 51 61 13 22 71 81 32 06 14 91 a1 b1 42 1.AQa."q.2.....B 314 23 24 15 52 c1 62 33 34 72 82 d1 43 07 25 92 53 #$.R.b34r..C.%.S 315 f0 e1 f1 63 73 35 16 a2 b2 83 26 44 93 54 64 45 ...cs5....&D.TdE 316 c2 a3 74 36 17 d2 55 e2 65 f2 b3 84 c3 d3 75 e3 ..t6..U.e.....u. 317 f3 46 27 94 a4 85 b4 95 c4 d4 e4 f4 a5 b5 c5 d5 .F'............. 318 e5 f5 56 66 76 86 96 a6 b6 c6 d6 e6 f6 37 47 57 ..Vfv........7GW 319 67 77 87 97 a7 b7 c7 d7 e7 f7 11 00 02 02 01 02 gw.............. 320 04 04 03 04 05 06 07 07 06 05 35 01 00 02 11 03 ..........5..... 321 21 31 12 04 41 51 61 71 22 13 05 32 81 91 14 a1 !1..AQaq"..2.... 322 b1 42 23 c1 52 d1 f0 33 24 62 e1 72 82 92 43 53 .B#.R..3$b.r..CS 323 15 63 73 34 f1 25 06 16 a2 b2 83 07 26 35 c2 d2 .cs4.%......&5.. 324 44 93 54 a3 17 64 45 55 36 74 65 e2 f2 b3 84 c3 D.T..dEU6te..... 325 d3 75 e3 f3 46 94 a4 85 b4 95 c4 d4 e4 f4 a5 b5 .u..F........... 326 c5 d5 e5 f5 56 66 76 86 96 a6 b6 c6 d6 e6 f6 27 ....Vfv........' 327 37 47 57 67 77 87 97 a7 b7 c7 ff da 00 0c 03 01 7GWgw........... 328 00 02 11 03 11 00 3f 00 f2 a5 3a ad 35 ba 40 0e ......?...:.5.@. 329 04 16 90 78 20 a8 25 07 94 aa d3 19 18 90 41 a2 ...x .%.......A. 330 13 9a 4b 9b b9 a0 91 c8 3d c8 ef a7 f2 14 46 35 ..K.....=.....F5 331 af fe 6c 6f f8 73 e3 3b 7e 92 6a ad 2c 30 75 64 ..lo.s.;~.j.,0ud 332 82 47 fd f9 a7 f3 5c 8a ec d7 b5 e4 d2 4b 79 0d .G....\......Ky. 333 73 a0 ba 3f f2 49 87 8b 61 4d 88 fd de 40 4a 66 s..?.I..aM...@Jf 334 51 fd e8 c7 e6 ff 00 03 f4 5a ee 63 d8 76 bd a5 Q........Z.c.v.. 335 a4 76 22 13 29 d9 75 b6 99 b1 ee 7c 71 b8 ca 82 .v".).u....|q... 336 78 be ad 79 70 f1 1e 1b e1 e9 c5 f3 29 24 92 49 x..yp.......)$.I 337 0a 49 24 92 52 92 45 c7 c4 bf 25 c5 b4 b7 76 d1 .I$.R.E...%...v. 338 2e 3c 00 3f 94 ef a2 d5 6f 33 a3 64 e1 63 7a f9 .<.?....o3.d.cz. 339 0f a9 85 c5 bb 29 f5 18 eb 1c 1c 1d b9 e2 ba 9c .....).......... 340 ff 00 63 36 fe 7a 69 c9 00 44 4c 87 11 da 3d 57 ..c6.zi..DL...=W 341 8c 59 0c 4c c4 4f 08 fd 2e 8d 3a da e7 1d 8d 11 .Y.L.O....:..... 342 22 75 47 ca fb 35 78 d5 d2 c2 1f 7c 87 58 f6 ea "uG..5x....|.X.. 343 06 91 e9 ef fc e4 1b 5f 4c 33 d1 05 a7 68 0f 27 ......._L3...h.' 344 b9 fc e8 42 4a ac 83 a8 ae 8c 9e e0 84 65 00 23 ...BJ........e.# 345 23 21 5c 7f 37 0c 7e 6f 47 f5 9f ff d0 f2 ae ca #!\.7.~oG....... 346 62 36 c1 3a 1f c0 84 cd 69 71 81 c9 47 a6 f6 e3 b6.:....iq..G... 347 3f 75 41 af 78 e1 ef 12 27 fe 0d bf f9 24 f1 3e ?uA.x...'....$.> 348 0d 40 e2 24 55 7f 15 f0 80 91 1c 52 e0 85 eb 2a .@.$U......R...* 349 e2 ff 00 16 3f a4 c2 fc 5c 8a 1a c7 da c2 c6 d9 ....?...\....... 350 f4 67 c9 05 5f ca ea 37 3c fa 77 1a b2 1b e2 01 .g.._..7<.w..... 351 81 3d 83 bd 8a 2e a8 67 39 b5 e0 63 90 e6 34 9b .=.....g9..c..4. 352 00 20 ff 00 68 f1 ec 67 ef a8 63 29 50 e3 00 5f . ..h..g..c)P.._ 353 51 f2 b3 f3 18 79 70 66 70 e5 26 30 e1 a8 65 1f Q....ypfp.&0..e. 354 ad 9c bf 4e 8e 3e 2c 5f f3 d8 62 f4 dc ac a6 ef ...N.>,_..b..... 355 a8 02 c0 40 73 8b 86 93 fb df 9c b6 3a 66 36 0d ...@s.......:f6. 356 6c 73 18 45 b7 6a 2c de ls.E.j,. 357 ######### 358 ``` 359 360 Above we specified `-X` to tell ngrep to treat the match expression as 361 hexadecimal, and `-x` to tell ngrep to print out the patterns it matches in 362 hexadecimal form. 363 364 As it turns out, several other packets also matched this pattern, but this 365 should give you a good idea of how to use hexadecimal patterns and the hex 366 output mode.