PacketFu version 0.2.0

PacketFu v0.2.0 was released today. There's really not a lot to this update, other than the direct inclusion of PcapRub and some more detailed installation instructions -- this week, a couple people wrote me to let me know that the installation instructions were, uh, less than forthcoming.

kballero wrote this Ubuntu forum post that goes into considerable detail on installing all the discrete components and some details on how to make wlan0 the default interface (as opposed to eth0). Many thanks for that!

At any rate, with this new version, I was able to install and run packetfu-shell.rb cleanly on a fresh LiveCD version of Back Track 3, so it should work for pretty much any Linux platform with a reasonbly recent libpcap version (If you get it running on WinXP and OS X, please let me know if you had to do anything special).

Still haven't worked out my performance problems; I suspect I'm going to have to ditch BinData entirely if I can't figure out how to fix it up to be a little more efficient with its recursion.

Working on PacketFu Performance

Here's a baseline of how PacketFu version 0.1.1 handles a set of 5000 packets. This benchmark test takes in a pcap file, then chucks all the processed packets into a Ruby array. The performance is horrid compared to Wireshark, but ignore that for a moment:

(these packets are all normal TCP packets)
(  1000) [17:26:48] {      20s}
(  2000) [17:27:13] {      25s}
(  3000) [17:27:43] {      30s}
(  4000) [17:28:19] {      36s}
(  5000) [17:29:01] {      42s}
2m33s elapsed, parsed 5001 packets.

Eek. So, the more packets I pull in, the slower PacketFu gets. This is pretty disastrous, if you're using PacketFu in offline mode.

So, after poking at PacketFu::Packet.parse() for a bit, I figured out this morning that if I make a good guess at the packet type before testing it for complete correctness, I get a fairly huge bonus in parsing speed. Here's a run with all normal TCP packets:

(  1000) [11:21:48] {      15s}
(  2000) [11:22:03] {      15s}
(  3000) [11:22:17] {      14s}
(  4000) [11:22:32] {      15s}
(  5000) [11:22:47] {      15s}
1m14s elapsed, parsed 5001 packets.

Testing the new and improved version with a mixed bag of packets, which contains ARP, TCP, ICMP, and UDP (and a few IPv6) packets:

(  1000) [12:21:15] {      11s}
(  2000) [12:21:30] {      15s}
(  3000) [12:21:48] {      18s}
(  4000) [12:22:10] {      22s}
(  5000) [12:22:36] {      26s}
1m32s elapsed, parsed 5001 packets.

Unfortunately, my creeping performance problem persists -- at least when I have a whole bunch of dissimilar packet types. But at least it's less pronounced now, and eliminated entirely when dealing with sets of TCP packets (which is going to be the most common use case, I figure).

---

Update: That was completely wrong. The only reason for the performance boost was that PacketFu::Packet.parse() was forgetting to read in the data. The below is even more true -- this is where the problem lies. Darnit! (please make sure to never use PacketFu r66, it's broken!)

---

tmanning has been looking at PacketFu lately as well, and believes that there are some (more) bugs in PacketFu::Packet.read(), mostly revolving around my atrocious design of how read() and parse() interrelate. I suspect this is the source of most of my performance problems as well, so keep an eye out for the next tagged version of PacketFu for some love in that part of the code. Oh, and I'll be fixing up the PacketFu::File.append() function to be a lot more sane, too.

PacketFu on Windows

Barely a week after I presented PacketFu, I got an idle question about PacketFu's usefulness on Windows.

Last night, while I was waiting for my brother and his family to drive up from Houston ahead of the Hurricane, I got to messing around with compiling PcapRub on Microsoft Vista and Windows XP. Much to my amazement, my goofy hacks worked. The rest of PacketFu is pure Ruby without C extensions or anything, so cross-platform love is already baked in there.

So, the latest revision of PacketFu seems to work fine on XP (I haven't tested the Vista machine yet). It works so well, in fact, that I duplicated the climax of my Lone Star RubyConf talk in the form of a Flash movie (link downloads the FLV, it's not embedded or anything).

How many other packet libraries have screencasts a week after they're built? ZERO, that's how many!