IPS vs WAF (or Four Things Your WAF Can’t Do)
A few days back someone tweeted a link to this blog post “WAF vs IPS (or Four Things Your IPS Can’t Do)“, and it waved a little red flag at me. Sorry bmestep (I didn’t see your full-name on your blog), you requested no flames but your post was begging for some follow-up. I will however try my best to make sure it’s not all flame and include some content as well.
“I see this often and I am always amused at the topic. I have worked with IDS/IPS for 8 years, so I know IPS when it was just a flavor of IDS that no one wanted to enable for fear of blocking access to users and customers. I chuckle at the thought of WAF being a glorified IPS.”
Err, yeah, me too i guess.
In fact I have work in a world where marketing spin and the FUD from *some* technology vendors has forced confusion between complementary technologies available on the market. Those who know me are most likely aware that I work for Sourcefire, so I guess I could be seen as part of the vendor problem but lets ignore that fact for the moment[1].
If you’re unlucky enough to walk around security trade-show floors, you’ll see a lot of similar messaging all around. Every year the message changes to what’s hot right now, but you can guarantee that most will be selling the same message with vastly different products.
This “me too” is where the pain comes from and Leon gets angry, let me spell things out.
- An IPS is not a WAF
- A WAF is not an IPS
- A Firewall is not an AV
- A BMW is not a DVR
- And to badly quote Mark Watson “I’m not interested in watching TV on my mobile phone, in the same way I’m not interested in taking a sh*t in my tumble dryer”
Maybe I should have titled this post “WAF vs Lawn Mower (or Four Things A WAF Can’t Do, That Your Lawn Mower Can).”
I don’t fancy sitting here tapping out a list of things that an IPS does that a WAF doesn’t because I don’t see that it has ANY relevance. Oh, and one last comment.
“As packets are inspected by an IPS, they are often discarded to improve performance. This is a key differentiator, because a WAF must retain packets in order to keep the context of a client web request and the subsequent server response.”
If there is anyone out there who wants to know how a good IPS works, you know, one that *doesn’t* discard packets to improve performance go grab the latest Snort tarball and start reading the source.
Anyway, I think it’s time I climb down off my soap-box.
[1] On a closing note, I feel that must mention that I think Sourcefire does a good job at marketing its IPS product. As a company we are not one to jump on the latest hype-cycle with some vapour-ware. Take a look at sourcefire.com, there’s no mention of our IPS being able to mow lawns.
-Leon
An update on the whole “car hits house” thing.
It’s been a while since I posted any updates to this blog, 2009 was a busy year both at work and at home.
For those of you who follow me on Twitter (@leonward) you may know of the great-crash of 2009. I’m not talking about Wall St here, I’m talking about when the front of my house met a car that somehow managed to drive off a straight road, across my neighbours garden, bounced off my wife’s car, and ended up partly in my lounge. At the time many people requested pictures and more info (“pics or it didn’t happen” springs to mind) but due to insurance claims and the police being involved I thought it best to keep things off the internet at the time.
As things are now on their way to being sorted out and yes, it has taken a while, I thought it safe to share a few of the pics of the carnage. Looking on the bright(er) side of things, at least we hadn’t completed the decorating / remodelling of the room (hence the awful original curtains, woodwork colour, no carpet etc). I also purchased a new amp and speaker package three days previous, but the shop didn’t have one of the items in stock so I didn’t pick it up. If it had come home with me the sub-woofer and a couple of speakers would have been a right-off. Annoyingly the re-plastering and painting of the walls was only completed two days before the crash.
So enjoy some Shadenfreude.
Sourcefire Quick Tip: Custom RNA Service Detection
As part of my Sourcefire “Quick Tip” series, a new video has been uploaded by our marketing guru’s to YouTube.
This time I ramble on about creating custom service detectors for Sourcefire’s RNA engine. This allows you to detect the network applications in use on your network and track them in the context of IPS events, real-time network change detection, Service and host white-listing as well as general network mapping.
I hope it’s of use to you out there. I haven’t got my next quick-tip subject planned yet, so if anyone has a suggestion please let me know.
Adding RNA Custom Services
If you would like to learn more about RNA, take a read here.
-Leon
Sourcefire Quick Tip: Host Input API and HippiEd
The on-line marketing team over at Sourcefire’s world domination HQ asked me to throw together some content for our Facebook / Youtube pages.
The result is a series of video quick tips showing some rarely talked about features of Sourcefire’s products. The first video is now available on the Youtube channel and Facebook page. The tool mentioned is available here.
HippiEd
Expect a few more over the next couple of weeks.
-Leon
Snoge and Dumbpig now on Google Code
I have moved the code repositories for DumbPig and Snoge over to google code.
Feel free to check the code out of subversion to look at the latest bleeding edge features.
DumbPig -> http://code.google.com/p/dumbpig
Snoge -> http://code.google.com/p/snoge/
ET RBN Blacklists with Snort and DumbPig
I spent a few minutes updating DumbPig to work with Marty’s latest blacklist patch, with some great results. It looks like Marty has done a great job in keeping packet performance high while providing a rich blacklist configuration. DumbPig for the as yet unenlightened, processes Snort rulesets and offers advice when a “dumb” rule is detected. Blacklist snort rules are a good example if dumbness so I thought I would focus in a bit on why and how to use these tools.
Before digging into the tech, let me tease you with some performance numbers. My test below was basic, but should provide some relation to the real world.
Test 1) Stock snort.conf and VRT subscription release (processing approx 1.2GB of pcap files)
$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf
Run time for packet processing was 46.994580 seconds
Snort processed 2752654 packets.
Test 2) Stock snort.conf, with the “emerging-rbn.rules” (20/july/2009) added
$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf
Run time for packet processing was 89.749301 seconds
Snort processed 2752654 packets.
Test 3) Stock snort.conf, but using a DumbPig created blacklist file from the same “emerging-rbn.rules”
$ snort –pcap-dir /var/local/pcaps/defcon-2004/ -A fast -l /tmp -c /etc/snort/snort.conf
Run time for packet processing was 48.348535 seconds
Snort processed 2752654 packets.
So that’s a whopper of a performance increase while maintaining the same IP based detection ability. I don’t claim you will see the same performance in the wild, but an increase like this should get your attention. Feel free to re-create the tests on your own network and let me know the results (for my interst only).
Configuring Snort 2.6.4.1 with the Blacklist patch (v2)
I have seen a few posts on the internet where people have run into issues configuring and using the blacklist patch. Below are the steps I took to build a system in prep for this weeks Snort Users webinar on DumbPig and other tools so I thought I would share the configuration here.
Starting platform: Debain Lenny – base installation + ssh and sudo (with lward in the sudoers)
1) Install the debs we know will be needed for the compile of Snort
lward@webexprep:~$ sudo apt-get install libpcap0.8 libpcap0.8-dev libdumbnet-dev build-essential libpcre3-dev automake autoconf libtool
2) Download and extract the snort 2.8.4.1 source
lward@webexprep:~$ wget http://dl.snort.org/snort-current/snort-2.8.4.1.tar.gz
lward@webexprep:~$ tar -zxvf ./snort-2.8.4.1.tar.gz
3) Download and extract the Blacklist patch
lward@webexprep:~$ cd snort-2.8.4.1/
lward@webexprep:~/snort-2.8.4.1$ wget http://www.snort.org/users/roesch/code/iplist.patch.v2.tgz
lward@webexprep:~/snort-2.8.4.1$ tar -zxvf ./iplist.patch.v2.tgz
4) Read the README.iplist
5) Pactch your Snort source tree
lward@webexprep:~/snort-2.8.4.1$ patch -p1 < iplist.patch
6) Rerun aclocal / automake/ autoconf
lward@webexprep:~/snort-2.8.4.1$ aclocal -I m4
lward@webexprep:~/snort-2.8.4.1$ automake
lward@webexprep:~/snort-2.8.4.1$ autoconf
7) Configure Snort (and enable IP listing).
lward@webexprep:~/snort-2.8.4.1$ ./configure –enable-iplist
(note there should be no errors , if you have m4 prelude messages, see the comment at the bottem of this post).
Compile / install
Debain / Ubuntu users: You will have to “fix” the dnet/dumbnet fsckup created by the decnet libraries poluting the debian package namespace. A simple symlink will suffice.
lward@webexprep:~/snort-2.8.4.1$ sudo ln -s /usr/include/dumbnet.h /usr/include/dnet.h
lward@webexprep:~/snort-2.8.4.1$ make
lward@webexprep:~/snort-2.8.4.1$ sudo make install
9) Set up your snort configuration.
lward@webexprep:~/snort-2.8.4.1$ sudo mkdir /etc/snort
lward@webexprep:~/snort-2.8.4.1$ sudo chown lward /etc/snort/
lward@webexprep:~/snort-2.8.4.1$ cp etc/* /etc/snort/
lward@webexprep:~/snort-2.8.4.1$ vi /etc/snort/snort.conf
(Change your RULE_PATH to /etc/snort/rules)
10) Grab the VRT snort ruleset from snort.org, and stuck it in your home directory. This exact process will depend on your subscription level.
11) Set up the VRT snort rules
lward@webexprep:~/snort-2.8.4.1$ cd
lward@webexprep:~$ pwd
/home/lward
lward@webexprep:~$ ls
snort-2.8.4.1 snort-2.8.4.1.tar.gz snortrules-snapshot-2.8.tar.gz
lward@webexprep:~$ tar -zxf ./snortrules-snapshot-2.8.tar.gz
lward@webexprep:~$ cp -r rules/ /etc/snort/
12) Grab yourself a pcap to test and play with
lward@webexprep:~$ wget http://rm-rf.co.uk/downloads/Honeynet-RFP-iis.tgz
lward@webexprep:~$ tar -zxvf ./Honeynet-RFP-iis.tgz
13) Test snort
lward@webexprep:~$ snort -c /etc/snort/snort.conf -A fast -l /tmp -r ~/Honeynet-RFP-iis.pcap
14) Enable, and test the blacklist functions
lward@webexprep:~$ echo preprocessor iplist: blacklist TestBlacklist /etc/snort/rules/test.blacklist >> /etc/snort/snort.conf
lward@webexprep:~$ echo 172.16.0.0/16 > /etc/snort/rules/test.blacklist
lward@webexprep:~/snort-2.8.4.1$ snort -c /etc/snort/snort.conf -A fast -l /tmp -r ~/Honeynet-RFP-iis.pcap
<Check that your are getting blacklist events in your /tmp/alert file. Make sure you add a CIDR that exists in your pcap test data!>
15) Install dumbpig
Install required perl modules from CPAN
lward@webexprep:~$ sudo cpan -e “Parse::Snort”
lward@webexprep:~$ sudo cpan -e “LWP::Simple”
lward@webexprep:~$ wget http://rm-rf.co.uk/downloads/dumbpig
lward@webexprep:~$ chmod +x ./dumbpig
16) Grab the latest Emerging threats RBN list
lward@webexprep:~$ wget http://www.emergingthreats.net/rules/emerging-rbn.rules
17) Convert the rule file into a blacklist
lward@webexprep:~$ ./dumbpig -q -r emerging-rbn.rules -b /etc/snort/rules/rbn.blacklist
DumbPig will detect rules that will work best in a blacklist, and add them to the file “rbn.blacklist”. For more usage inforamtion, take a look at the dumbpig page.
lward@webexprep:~$ head -n 5 /etc/snort/rules/rbn.blacklist
# Autogenerated blacklist by DumbPig from emerging-rbn.rules
# Contact leon.ward@sourcefire.com
# For more information about dumbPig visit http://rm-rf.co.uk
114.80.67.30/32 114.80.67.32/32 <snip> # From Sid 2406000 : “ET RBN Known Russian Business Network IP TCP (1)” : emerging-rbn.rules
114.80.67.30/32 114.80.67.32/32 <snip> # From Sid 2406001 : “ET RBN Known Russian Business Network IP UDP (1)” : emerging-rbn.rules
18) Reconfigure your snort.conf to the the rbn.blacklist
Something like this should work for you” “preprocessor iplist: blacklist RBN_Hosts /etc/snort/rules/rbn.blacklist”
And that’s it, the rest is up to you to use/abuse as you need.
Troubleshooting
Troubleshooting build problems. If you are having fun with one of the below, start with a clean source tree, re-patch, and follow step 6 (note the aclocal command).
undefined reference to `SetupIpList' configure.in:1050: warning: macro `AM_PATH_LIBPRELUDE' not found in library Happy Snorting - Leon
DumbPig 0.5 Updates, bug fixes, and real-word usage
The original release of DumbPig seemed to be well received, and promoted a load of quality feedback. I found some time earlier to take action on the first load of requests, and advice so please provide a warm welcome to version 0.5.
Changes in this version.
- Better support for rules copy/pasted out of the Sourcefire Defense Center GUI
- (Right click on a rule, and go to documentation to see it’s source) (thanks @tomdixn)
- Removal of the mega if/then/else block (Nigel beat me into submission)
- Support for some forgotten keywords
- Censor ability to hide the sensitive parts of private rules
- Check for updates over the internet
- Addition of some new keywords (thanks @tryke)
- Better whitespace normalization
I hope Dumbpig could become a useful tool for the community, Documentation and code are available on the Dumbpig page.
DumbPig – Automated checking for Snort rulesets
From time to time I have the honour of working with “special” snort rule-sets, ones that I’m not allowed to analyse or take away. I understand that some like to keep their detection closed, but when they have errors in them that ….
- Prevent an import into the Sourcefire Defense Center
- Cause detection to run like a pig
- Incorrectly assign event priority
- Behave in a completely different way to how the “expert writer” expected them to
… I have to sit up and bitch for a while. I also see the same problems again and again in public rule sets, so I wanted to make a way to automate my way out of pain.
Introducing DumbPig, “Because your rules are not so good actually”.
DumbPig parses a snort rule-set, and depending on command line options, will recommend “fixes” for dumb Snort rules.
Take these two dumb rules as examples:
alert ip any any -> any 53 (msg: “I found a scary DNS name woooo”; content: “woot”; sid: 1;)
drop tcp 1.1.1.1 any -> $HOME_NET any (msg: “Communication with nasty host”; sid: 2; rev:1;)
Sid:1 above has some obvious problems for use in the real-world
- The IP protocol header doesn’t have port numbers, it is common to see someone write a rule like this in an attempt to search for content in both TCP and UDP data. It is faster and correct to create two rules, one for TCP and one for UDP.
- There isn’t a revision number, this makes it hard to track the rule through it’s evolution. This is required for importing to a Sourcefire Defense Center along with some other tools.
- There is no classificaion data to equate what the discovery of this rule means to the user
- There is no prioritization specification (partially due to the above)
Sid:2 Also has some other issues
- It doesn’t do any deep-packet checks, what’s the point in wasting IPS cycles to do what a firewall or router can do much faster. This being said, Marty has recently added a Snort preprocessor for blacklisting, this rule would be a good candidate for use in this preproc.
[20:30:48]lward@drax~/Documents/Sourcefire/Tools/dumbpig$ ./dumbpig.pl Usage dumbPig <args> -r or --rulefile <rulefile> -s or --sensitivity <1-3> Sensitivity level, Higher the number, the higher the pass-grade -b or --blacklist Enable blacklist output (see Marty's Blog post for details) -p or --pause Pause for ENTER after each FAIL -w or --write Filename to wite CLEAN rules to -q or --quiet Suppress FAIL, only provide summary
It’s simple to use, just give it a rule file and inspect the output report. It also presents the rule in Snort multi-line format (escaped line breaks) to make it easy to read when things get complicated like the
[20:28:15]lward@drax~/Documents/Sourcefire/Tools/dumbpig$ ./dumbpig.pl -r dumb.rules -s 3 -b
DumbPig version 0.1 - leon.ward@sourcefire.com
Because I hate looking for the same dumb problems with "Private" snort rule-sets
__,, ( Dumb-pig says )
~( oo ---( "ur rulz r not so )
'''' ( gud akshuly" * )
Config
----------------------
* Sensitivity level - 3/3
* Blacklist output ENABLED
* Processing File - dumb.rules
* Pause - 0
* Output clean rules to - 0
* Quite mode = 0
--- [FAIL] 1 on line 1 (dumb.rules) -----------------------------
Reasons:
- IP rule with port number (or var that could be set to a port number).
This is BAD and invalid syntax.
The IP protocol doesn't have port numbers.
If you want to inspect both UDP and TCP traffic on specific ports use two rules,
its faster and valid syntax.
- No revision number! Please add a rev: keyword
- No classification specified - Please add a classtype to add correct priority rating
alert ip any any -> any 53 (
msg: "I found a scary DNS name woooo"; \
content: "woot"; \
sid: 1; \
)
# alert ip any any -> any 53 (msg: "I found a scary DNS name woooo"; content: "woot"; sid: 1;)
--- [FAIL] 2 on line 2 (dumb.rules) -----------------------------
Reasons:
- No classification specified - Please add a classtype to add correct priority rating
- No deep packet checks? This rule looks more suited to a firewall or blacklist
- IP rule without a content match. Put this in a firewall!
drop ip 1.1.1.1 any -> $HOME_NET any (
msg: "Communication with nasty host found"; \
sid: 2; \
rev: 1; \
)
# drop ip 1.1.1.1 any -> $HOME_NET any (msg: "Communication with nasty host found"; sid: 2; rev:1;)
----Blacklist----
Add this lines to your snort.conf
preprocessor iplist: blacklist { 1.1.1.1/32 }
--------------------------------------
Total: 2 fails over 2 lines in dumb.rules
- Contact leon.ward@sourcefire.com
Think of this as a preview release, give me feedback and find me bugs!
Download dumbpig here.
-Leon
Thoughts of the cloud – Part one of …… some.
I (via Sourcefire) recently had the opportunity to present at a could computing security event in Ireland organised by Calyx, unfortunately for me the cloud-specific focus of the event managed to slip past both myself and my colleagues in marketing unnoticed until the day before the event. This forced me to take the audience on an unexpected thirty minute tangent away from the cloud centric content they were expecting (sorry about that), but while I wasn’t talking I was given a great opportunity to witness the confusion that is presented as “The Cloud”.
Clearly cloud-stuff is getting more and more headlines, especially when you link the “C” word together with security.
Each presenter provided a slightly different definition of what cloud computing meant to them (and their marketing departments), but somehow they all managed to agree on key security risks associated with suggestions of …….
- Dumping mission critical internally hosted services and throwing them into the cloud
- Placing your “trusted data” in the trust of another “untrusted” party
- Believing that a cloud provider (SAAS, PAAS, IAAS etc) has a magic ability to look after their systems better than your people look after your systems
- Burying you head in the sand while shouting “la la la don’t remind me that I’m still accountable for all of this with little to no control!”
Quite correctly none of the presenting vendors offered a silver bullet to solve these issues, and if they had offered one up I’m sure someone in the audience would have called them a liar. Instead they provided intelligent thoughts and opinions to potential workarounds, however I don’t remember them touching on what I see as an important and overlooked point.
Cloud technology will rarely provide a complete migration path, it is most likely to be a technology addition. I won’t go into the justification of my point of view right now, but believe me this is the way I see it. This also means that my non-cloud “Know your network better than your enemy” tangent that I took the audience on is still important. I hope the “C” word moves further along the hype-cycle path to sit somewhere better understood soon.
Earlier today I read Amrit Williams’ wonderfully sarcasm-rich post on a similar subject, I relate to it because it touches on my views of cloud addition. Reading it also reminded me to hit the publish button on this posts draft.
-Leon
TweetYard – Sourcefire and Snort alerts to Twitter
Update: – You can download tweetyard here.
There has been some of discussion of late on the snort-* lists of late regarding Unified alerting vs direct DB access.
I stopped storing events in a DB years back when I stopped using ACID (and yes that was back-in-the-day before BASE came into being). My personal Snort requirements are pretty simple and fast output has always worked well for me linked with a load of swatch-foo and custom perl scripts. After hanging my head in shame for not converting to unified yet (cobblers children clearly have no shoes over here) I thought it would be wise to put some effort in.
I used to receive all of my Snort IDS events via email, but email is *so* web 1.0. So I thought I would hook into Twitter for real-time alerting 
So far, so good, and it only took about an hour to build. Kudos to Jason Brvenik for his snort-unified.pm and sample barnyard replacement, it was a good base for what I wanted to hack together. Because I put this together more than fun more than anything else, feel free to follow a censored Twitter feed of my IPS events (If you didn’t have enough to deal with already). I have blanked the IPs of my protected systems in an attempt to raise the smarts-to-abuse bar up 0.2 inches above short skiddie tall.
I will upload the code when I get a spare couple of minutes, but as I will be attached to the Sourcefire booth @ Infosecurity London for the next three days it may take a while. Hooking it into Sourcefire’s Estreamer is also on the cards the next time I get some down-time.
If anyone is at the show, feel free to drop by the Sourcefire booth and say hi (and to bring me a Coffee at the same time).
-Leon
