An alchemists view from the bar

Network Security Alchemy

Posts Tagged ‘DumbPig

ET RBN Blacklists with Snort and DumbPig

leave a comment »

I spent a few minutes updating DumbPig to work with Marty’s latest blacklist patch, with some great results. It looks like Marty has done a great job in keeping packet performance high while providing a rich blacklist configuration. DumbPig for the as yet unenlightened, processes Snort rulesets and offers advice when a “dumb” rule is detected. Blacklist snort rules are a good example if dumbness so I thought I would focus in a bit on why and how to use these tools.

Before digging into the tech, let me tease you with some performance numbers. My test below was basic, but should provide some relation to the real world.

Test 1) Stock snort.conf and VRT subscription release (processing approx 1.2GB of pcap files)

$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf

Run time for packet processing was 46.994580 seconds
Snort processed 2752654 packets.

Test 2) Stock snort.conf, with the “emerging-rbn.rules” (20/july/2009) added

$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf

Run time for packet processing was 89.749301 seconds
Snort processed 2752654 packets.

Test 3) Stock snort.conf, but using a DumbPig created blacklist file from the same “emerging-rbn.rules”

$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf

Run time for packet processing was 48.348535 seconds
Snort processed 2752654 packets.

So that’s a whopper of a performance increase while maintaining the same IP based detection ability. I don’t claim you will see the same performance in the wild, but an increase like this should get your attention. Feel free to re-create the tests on your own network and let me know the results (for my interst only).

Configuring Snort 2.6.4.1 with the Blacklist patch (v2)

I have seen a few posts on the internet where people have run into issues configuring and using the blacklist patch. Below are the steps I took to build a system in prep for this weeks Snort Users webinar on DumbPig and other tools so I thought I would share the configuration here.

Starting platform: Debain Lenny – base installation + ssh and sudo (with lward in the sudoers)

1) Install the debs we know will be needed for the compile of Snort

lward@webexprep:~$ sudo apt-get install libpcap0.8 libpcap0.8-dev libdumbnet-dev build-essential libpcre3-dev automake autoconf libtool

2) Download and extract the snort 2.8.4.1 source

lward@webexprep:~$ wget http://dl.snort.org/snort-current/snort-2.8.4.1.tar.gz
lward@webexprep:~$ tar -zxvf ./snort-2.8.4.1.tar.gz

3) Download and extract the Blacklist patch

lward@webexprep:~$ cd snort-2.8.4.1/
lward@webexprep:~/snort-2.8.4.1$ wget http://www.snort.org/users/roesch/code/iplist.patch.v2.tgz
lward@webexprep:~/snort-2.8.4.1$ tar -zxvf ./iplist.patch.v2.tgz

4) Read the README.iplist

5) Pactch your Snort source tree

lward@webexprep:~/snort-2.8.4.1$ patch -p1 < iplist.patch

6) Rerun aclocal / automake/ autoconf

lward@webexprep:~/snort-2.8.4.1$ aclocal -I m4
lward@webexprep:~/snort-2.8.4.1$ automake
lward@webexprep:~/snort-2.8.4.1$ autoconf

7) Configure Snort (and enable IP listing).

lward@webexprep:~/snort-2.8.4.1$ ./configure –enable-iplist

(note there should be no errors , if you have m4 prelude messages, see the comment at the bottem of this post).

8) Compile  / install

Debain / Ubuntu users: You will have to “fix” the dnet/dumbnet fsckup created by the decnet libraries poluting the debian package namespace. A simple symlink will suffice.

lward@webexprep:~/snort-2.8.4.1$ sudo ln -s /usr/include/dumbnet.h /usr/include/dnet.h
lward@webexprep:~/snort-2.8.4.1$ make
lward@webexprep:~/snort-2.8.4.1$ sudo make install

9) Set up your snort configuration.

lward@webexprep:~/snort-2.8.4.1$ sudo mkdir /etc/snort
lward@webexprep:~/snort-2.8.4.1$ sudo chown lward /etc/snort/
lward@webexprep:~/snort-2.8.4.1$ cp etc/* /etc/snort/
lward@webexprep:~/snort-2.8.4.1$ vi /etc/snort/snort.conf

(Change your RULE_PATH to /etc/snort/rules)

10) Grab the VRT snort ruleset from snort.org, and stuck it in your home directory. This exact process will depend on your subscription level.

11) Set up the VRT snort rules

lward@webexprep:~/snort-2.8.4.1$ cd
lward@webexprep:~$ pwd
/home/lward
lward@webexprep:~$ ls
snort-2.8.4.1  snort-2.8.4.1.tar.gz  snortrules-snapshot-2.8.tar.gz
lward@webexprep:~$ tar -zxf ./snortrules-snapshot-2.8.tar.gz
lward@webexprep:~$ cp -r rules/ /etc/snort/

12) Grab yourself a pcap to test and play with

lward@webexprep:~$ wget http://rm-rf.co.uk/downloads/Honeynet-RFP-iis.tgz
lward@webexprep:~$ tar -zxvf ./Honeynet-RFP-iis.tgz

13) Test snort

lward@webexprep:~$ snort -c /etc/snort/snort.conf -A fast -l /tmp -r ~/Honeynet-RFP-iis.pcap

14) Enable, and test the blacklist functions

lward@webexprep:~$ echo preprocessor iplist: blacklist TestBlacklist /etc/snort/rules/test.blacklist >> /etc/snort/snort.conf
lward@webexprep:~$ echo 172.16.0.0/16 > /etc/snort/rules/test.blacklist
lward@webexprep:~/snort-2.8.4.1$ snort -c /etc/snort/snort.conf -A fast -l /tmp -r ~/Honeynet-RFP-iis.pcap

<Check that your are getting blacklist events in your /tmp/alert file. Make sure you add a CIDR that exists in your pcap test data!>

15) Install dumbpig

Install required perl modules from CPAN

lward@webexprep:~$ sudo cpan -e “Parse::Snort”
lward@webexprep:~$ sudo cpan -e “LWP::Simple”
lward@webexprep:~$ wget http://rm-rf.co.uk/downloads/dumbpig
lward@webexprep:~$ chmod +x ./dumbpig

16) Grab the latest Emerging threats RBN list

lward@webexprep:~$ wget http://www.emergingthreats.net/rules/emerging-rbn.rules

17) Convert the rule file into a blacklist

lward@webexprep:~$ ./dumbpig -q -r emerging-rbn.rules -b /etc/snort/rules/rbn.blacklist

DumbPig will detect rules that will work best in a blacklist, and add them to the file “rbn.blacklist”. For more usage inforamtion, take a look at the dumbpig page.

lward@webexprep:~$ head -n 5 /etc/snort/rules/rbn.blacklist
# Autogenerated blacklist by DumbPig from emerging-rbn.rules
# Contact leon.ward@sourcefire.com
# For more information about dumbPig visit http://rm-rf.co.uk
114.80.67.30/32 114.80.67.32/32 <snip>    # From Sid 2406000 : “ET RBN Known Russian Business Network IP TCP (1)” : emerging-rbn.rules
114.80.67.30/32 114.80.67.32/32 <snip>    # From Sid 2406001 : “ET RBN Known Russian Business Network IP UDP (1)” : emerging-rbn.rules

18) Reconfigure your snort.conf to the the rbn.blacklist

Something like this should work for you” “preprocessor iplist: blacklist RBN_Hosts /etc/snort/rules/rbn.blacklist”

And that’s it, the rest is up to you to use/abuse as you need.

Troubleshooting

Troubleshooting build problems. If you are having fun with one of the below, start with a clean source tree, re-patch, and follow step 6 (note the aclocal command).

undefined reference to `SetupIpList'
configure.in:1050: warning: macro `AM_PATH_LIBPRELUDE' not found in library

Happy Snorting - Leon
Advertisements

Written by leonward

July 20, 2009 at 12:06 pm

Posted in Security, snort

Tagged with ,

DumbPig 0.5 Updates, bug fixes, and real-word usage

leave a comment »

The original release of DumbPig seemed to be well received, and promoted a load of quality feedback. I found some time earlier to take action on the first load of requests, and advice so please provide a warm welcome to version 0.5.

Changes in this version.

  • Better support for rules copy/pasted out of the Sourcefire Defense Center GUI
    • (Right click on a rule, and go to documentation to see it’s source) (thanks @tomdixn)
  • Removal of the mega if/then/else block (Nigel beat me into submission)
  • Support for some forgotten keywords
  • Censor ability to hide the sensitive parts of private rules
  • Check for updates over the internet
  • Addition of some new keywords (thanks @tryke)
  • Better whitespace normalization

I hope Dumbpig could become a useful tool for the community, Documentation and code are available on the Dumbpig page.

Written by leonward

June 24, 2009 at 1:34 pm

Posted in Security, snort

Tagged with

DumbPig – Automated checking for Snort rulesets

with 3 comments

From time to time I have the honour of working with “special” snort rule-sets, ones that I’m not allowed to analyse or take away. I understand that some like to keep their detection closed, but when they have errors in them that ….

  • Prevent an import into the Sourcefire Defense Center
  • Cause detection to run like a pig
  • Incorrectly assign event priority
  • Behave in a completely different way to how the “expert writer” expected them to

… I have to sit up and bitch for a while. I also see the same problems again and again in public rule sets, so I wanted to make a way to automate my way out of pain.

Introducing DumbPig, “Because your rules are not so good actually”.

DumbPig parses a snort rule-set, and depending on command line options, will recommend “fixes” for dumb Snort rules.

Take these two dumb rules as examples:

alert ip any any -> any 53 (msg: “I found a scary DNS name woooo”; content: “woot”; sid: 1;)

drop tcp 1.1.1.1 any -> $HOME_NET any (msg: “Communication with nasty host”; sid: 2; rev:1;)

Sid:1 above has some obvious problems for use in the real-world

  • The IP protocol header doesn’t have port numbers, it is common to see someone write a rule like this in an attempt to search for content in both TCP and UDP data. It is faster and correct to create two rules, one for TCP and one for UDP.
  • There isn’t a revision number, this makes it hard to track the rule through it’s evolution. This is required for importing to a Sourcefire Defense Center along with some other tools.
  • There is no classificaion data to equate what the discovery of this rule means to the user
  • There is no prioritization specification (partially due to the above)

Sid:2 Also has some other issues

  • It doesn’t do any deep-packet checks, what’s the point in wasting IPS cycles to do what a firewall or router can do much faster. This being said, Marty has recently added a Snort preprocessor for blacklisting, this rule would be a good candidate for use in this preproc.
[20:30:48]lward@drax~/Documents/Sourcefire/Tools/dumbpig$ ./dumbpig.pl

Usage dumbPig <args>
   -r or --rulefile  <rulefile>
   -s or --sensitivity <1-3> Sensitivity level, Higher the number, the higher the pass-grade
   -b or --blacklist       Enable blacklist output (see Marty's Blog post for details)
   -p or --pause           Pause for ENTER after each FAIL
   -w or --write           Filename to wite CLEAN rules to
   -q or --quiet           Suppress FAIL, only provide summary

It’s simple to use, just give it a rule file and inspect the output report. It also presents the rule in Snort multi-line format (escaped line breaks) to make it easy to read when things get complicated like the

[20:28:15]lward@drax~/Documents/Sourcefire/Tools/dumbpig$ ./dumbpig.pl -r dumb.rules -s 3 -b

DumbPig version 0.1 - leon.ward@sourcefire.com
Because I hate looking for the same dumb problems with "Private" snort rule-sets

  __,,    ( Dumb-pig says     )  
~(  oo ---( "ur rulz r not so )
  ''''    ( gud akshuly" *    )   

Config
----------------------
* Sensitivity level - 3/3
* Blacklist output ENABLED
* Processing File - dumb.rules
* Pause - 0
* Output clean rules to - 0
* Quite mode = 0

--- [FAIL] 1 on line 1 (dumb.rules) -----------------------------
Reasons:
 - IP rule with port number (or var that could be set to a port number).
   This is BAD and invalid syntax.
   The IP protocol doesn't have port numbers.
   If you want to inspect both UDP and TCP traffic on specific ports use two rules,
   its faster and valid syntax.
 - No revision number! Please add a rev: keyword
 - No classification specified - Please add a classtype to add correct priority rating

alert ip any any -> any 53  (
 msg:  "I found a scary DNS name woooo"; \
 content:  "woot"; \
 sid:  1; \
 )
# alert ip any any -> any 53 (msg: "I found a scary DNS name woooo"; content: "woot"; sid: 1;)

--- [FAIL] 2 on line 2 (dumb.rules) -----------------------------
Reasons:
 - No classification specified - Please add a classtype to add correct priority rating
 - No deep packet checks? This rule looks more suited to a firewall or blacklist
 - IP rule without a content match. Put this in a firewall!

drop ip 1.1.1.1 any -> $HOME_NET any  (
 msg:  "Communication with nasty host found"; \
 sid:  2; \
 rev: 1; \
 )
# drop ip 1.1.1.1 any -> $HOME_NET any (msg: "Communication with nasty host found"; sid: 2; rev:1;)

----Blacklist----
Add this lines to your snort.conf
preprocessor iplist: blacklist { 1.1.1.1/32  }
--------------------------------------
Total: 2 fails over 2 lines in dumb.rules
- Contact leon.ward@sourcefire.com

Think of this as a preview release, give me feedback and find me bugs!

Download DumbPig here
-Leon

Written by leonward

June 7, 2009 at 8:03 pm

Posted in Security, snort, Sourcefire

Tagged with