An alchemists view from the bar

Network Security Alchemy

Archive for July 2008

Snort 3 Beta on Ubuntu / Debian Installation

with 11 comments

A few days ago I had some spare(ish) time, and decided to take a look at the Snort 3.x beta. I took some time looking at the alpha release in 07, and am happy to see how far it has come since then.

Over the last few weeks, I have seen a couple of posts to the Snort forums asking for help to get Snort 3.x up and running. It is good to see that others are interested in testing the engine, and unfortunate that there is such a steep learning curve to get to grips with the new way that Snort, and the Snort Security Platform now work. I have a hunch that after a little effort in learning the new methods it will all soon seem like second nature to all of us.

I thought I would share the steps I went through to get Snort 3 running on a test VMware virtual machine in the hope they can help out others.

My base OS is Ubuntu jeos, a stripped down build of Ubuntu designed and optimised for running in a VMware instance, the below instructions should work for pretty much any Debian based OS and let me know if they don’t!

The Jeos installation leaves me with a minimal Ubuntu system, comparable to Debian “base” , so to build anything on top of this we need to install some extra packages.

Before we try to install and configure the Snort Security Platform along with the Snort 3 analitical engine, lets make sure that we are able to get snort 2.8.2.1 (the latest stable 2.x release at the time of writing) working on our device. This extra task will save us a LOT of time later.

Building and installing Snort 2.8

Firstly I want to access this device via ssh, so a ssh daemon is required along with some other basic tools

sudo apt-get install ssh wget

We need all the key components to allow us to compile code, the build-essential meta-package will install all of these for me.

sudo apt-get install build-essential

To build Snort from source, we need to install some key libraries and development headers that it requires. libpcap is the promiscuous packet capture library, it is used by Snort, wireshark, tcpdump etc to capture network traffic.

sudo apt-get install libpcap0.8 libpcap0.8-dev

Snort supports PCRE for matching data within packets and data streams, therefore we need to install the required libraries and header files.

sudo apt-get install libpcre3 libpcre3-dev

Once Snort’s dependancies are installed, lets get the snort 2.x source and install it.

wget http://snort.org/dl/current/snort-2.8.2.1.tar.gz
tar -zxf ./snort-2.8.2.1.tar.gz
cd snort-2.8.2.1
./configure
make
sudo make install
sudo mkdir /etc/snort
sudo cp etc/* /etc/snort

We should now be in a position where Snort 2.8.x is ready to be configured for use, lets check its availability with a snort -V to check.

snort -V

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.8.2.1 (Build 16)
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
           (C) Copyright 1998-2008 Sourcefire Inc., et al.
           Using PCRE version: 7.4 2007-09-21

Before we can test Snort in any way, we need a few more things, some rules, and some test data. How you access the Snort rulebase is dependant on whether or not you are a Snort rule subscriber, and what level of subscription you have. for this simple test we don’t need the latest and greatest rules from the Sourcefire VRT (Vulnerablity Researh Team) like if we were running a real sensor, but we need a modern set of rules that will work with a 2.8 engine.

Go and register an account on snort.org, and download the “registered user release”, or use whatever ruleset you have handy for a 2.8 engine. Put the rule files into /etc/snort/rules/

<get hold of rule tarball>
tar -zxf snortrules-snapshot-CURRENT.tar.gz
sudo cp -r rules/ /etc/snort/

We now need to set the “RULE_PATH” variable in /etc/snort/snort.conf to point to /etc/snort/rules. I use vi to acomplish this.

sudo vi /etc/snort/snort.conf

After editing, the line should look like this

grep "var RULE_PATH" /etc/snort/snort.conf
var RULE_PATH /etc/snort/rules

Lets not give snort a test

snort -c /etc/snort/snort.conf -A fast -l /tmp -T

This command tells snort to start up in IDS mode reading /etc/snort/snort.conf. The output mode is “Fast”, logging will be to the /tmp directory, and to simply test the config and exit.

You should see an output a little like this:

        --== Initialization Complete ==--

   ,,_     -*> Snort! <*-
  o"  )~   Version 2.8.2.1 (Build 16)
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
           (C) Copyright 1998-2008 Sourcefire Inc., et al.
           Using PCRE version: 7.4 2007-09-21

           Rules Engine: SF_SNORT_DETECTION_ENGINE  Version 1.8  <Build 14>
           Preprocessor Object: SF_DCERPC  Version 1.1  <Build 4>
           Preprocessor Object: SF_FTPTELNET  Version 1.1  <Build 10>
           Preprocessor Object: SF_SMTP  Version 1.1  <Build 7>
           Preprocessor Object: SF_Dynamic_Example_Preprocessor  Version 1.0  <Build 1>
           Preprocessor Object: SF_SSH  Version 1.1  <Build 1>
           Preprocessor Object: SF_DNS  Version 1.1  <Build 2>
           Preprocessor Object: SF_SSLPP  Version 1.0  <Build 1>

Snort successfully loaded all rules and checked all rule chains!
Snort exiting

The below pcap is one I commonly use for testing an installation, it contains some obvious attacks from about 2001. I host it here to make it easy for me to find but its originally from the honeynet project (original data captured by Rain Forest Puppy).

cd /tmp
wget rm-rf.co.uk/downloads/Honeynet-RFP-iis.tgz
tar -zxvf ./Honeynet-RFP-iis.tgz

Now we have Snort configured (using the term losely), and a pcap to test snort with, lets give it a run.

snort -c /etc/snort/snort.conf -A fast -l /tmp -r ./Honeynet-RFP-iis.pcap

If successful you should have a file in /tmp/Alert that contains lots of alarms, and /tmp/snort.log.<timestamp> that contains the pcaps of the detected events.

If you do, lets move on to building and installing snortsp.

Building Snortsp 3.0Beta

The Snort security platform has other requirements for building on top of the above that were needed for Snort 2.x

Libnet and libdumbnet provide low level packet creation and modification libraries. Note that libdumbnet is the Debian name equivalent of libdnet in other distributions. The curses libraries handle screen and terminal manipulation, Libreadline provides history and tab completion for terminal commands to improve the user interaction expience with a shell. Lua is the new scripting language used in the Snort Security Platform, flex and bison are more modern replacements to lex and yacc. A UUID (universally unique identifier) generator is also now required for SnortSP.

sudo apt-get install libnet1 libnet1-dev \
    libdumbnet-dev libdumbnet1 \
    libncurses5 libncurses5-dev \
    libreadline5 libreadline5-dev \
    liblua5.1-0 liblua5.1-0-dev \
    flex bison \
    uuid uuid-dev

Now download and compile SnortSP.

Note: At the time of writing snort 3.0.0b2 is the most current release. Don’t use old betas, go grab the latest from snort.org.

cd ~
wget http://www.snort.org/dl/prerelease/3.0.0-b2/snortsp-3.0.0b2.tar.gz
tar -zxf ./snortsp-3.0.0b2.tar.gz
cd snortsp-3.0.0b2
./configure
make
sudo make install
sudo ldconfig
sudo mkdir /etc/snortsp
sudo cp etc/* /etc/snortsp/

Now SnortSP should be installed, not that this is just the security platform and not the snort engine itself. Snort, the analytical engine, needs to be built separately. Before we compile it first check that snortsp works

snortsp -V
SnortSP Version 3.0.0b2

cd src/analysis/snort/

./configure --with-platform-includes=/usr/local/include/snortsp/ \
    --with-platform-libraries=/usr/local/lib/snortsp/
make
sudo make install

The snort engine should now be ready for configuration and use under SnortSP. The challenge we have now it to get it doing what we want.

Start up snortsp to check the platform it is ready for use, (ssp.shutdown() is the command to shutdown the snortsp shell)

sudo snortsp -L /etc/snortsp/snort.lua
[+] Loaded pcap DAQ
[+] Loaded file DAQ
[+] Loaded afpacket DAQ
[*] DAQ Modules Loaded...
[*] Loading decoder modules
[+] Loaded ethernet
[+] Loaded null
[+] Loaded arp
[+] Loaded ip
[+] Loaded tcp
[+] Loaded udp
[+] Loaded icmp
[+] Loaded icmp6
[+] Loaded gre
[+] Loaded mpls
[+] Loaded 8021q
[+] Loaded ipv6
[+] Loaded ppp
[+] Loaded pppoe
[+] Loaded gtp
[+] Loaded raw
[*] Decoder initialized...
[*] Flow manager initialized...
[*] Data source subsystem loaded
[*] Engine manager initialized
Control thread running - 3083479952 (22010)
[*] Loading command interface
[!] Loading SnortSP command metatable
[!] Loading data source command metatable
[!] Loading engine command metatable
[!] Loading output command metatable
[!] Loading analyzer command metatable
Executing /etc/snortsp/snort.lua
   ,,_     -*> SnortSP! <*-
  o"  )~   Version 3.0.0b2 (Build 9) [BETA]
   ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
           (C) Copyright 2008 Sourcefire Inc.
snort> ssp.shutdown()

Because snortsp is a radically new method of handling data sources and detection engines (such as the Snort analytic), some tools have been provided within the snortsp tarball for porting your old method of starting up snort and having it run within the snortsp. This tool is called sspiffy.sh. This tool was a key element to me getting my first instance of snort inside the snortsp running the packets contained within my pcap through detection, however it wasn’t the simple walk in the park it was supposed to be.

I suggest you take a look at the documentation for this tool and see how you get on, however expect the lua file that it creates to not be perfect, but close. Also make sure that it has write access to your snort.conf. With this in mind, i decided to share my sightly modified lua file, based on the output of sspiffy.sh. It works for me along with this snort.conf. Feel free to hack about with it to make it do what you want.

My snort.lua file (save it to /etc/snort)

My snort3 beta snort.conf file (save it to /etc/snort)

cd /tmp/
wget http://rm-rf.co.uk/downloads/snort3_beta_pcap.lua
sudo cp snort3_beta_pcap.lua /etc/snort/

sudo mv /etc/snort/snort.conf /etc/snort/snort.conf.2.8
wget http://rm-rf.co.uk/downloads/snort3_beta.conf
sudo cp /tmp/snort3_beta.conf /etc/snort/snort.conf

Now lets fire up snortsp using the lua file above, and see how she goes. If successful you should see output like this.

Anyway, I need to spend some more time playing with the tool and less writing all of this. Let me know if I have got something wrong, or if these instructions don’t work for you.

Happy Snortin’

-Leon

Advertisements

Written by leonward

July 27, 2008 at 5:45 pm

Posted in Security

Tagged with , ,

“Not using PCAP_FRAMES”, A.k.a When good verbosity goes bad

with 26 comments

The same questions get posted again and again to the Snort forums, at the moment this is the most frequently misunderstood, and asked question that catches my eye.

Help !!!!!!!!
Snort doesn’t work !
It dies with a “Not Using PCAP_FRAMES” error message”.
Quick, quick help me now!

I’m ranting about this here so hopefully when people google the “Not using PCAP_FRAMES” message before blindly posting to the forums or mail lists for help (I know, I can dream), maybe this post will appear in their search results solving their non-issue.

What is an error message?

Lets look a real error messages first, unlike the above.

--== Initializing Snort ==--
Initializing Output Plugins!
Initializing Preprocessors!
Initializing Plug-ins!
Parsing Rules file /this/rules/file/does/not/exist
ERROR: Unable to open rules file: /this/rules/file/does/not/exist or /this/rules/file/does/not//this/rules/file/does/not/exist
Fatal Error, Quitting..

The error condition above is clearly identified, other messages not prefixed with “ERROR” are supporting messages to help a user understand what the system is doing. This rule holds true with most software and not Snort alone.

What is a PCAP_FRAME?

PCAP_FRAMES is an environment variable that is used to pass a configuration setting to a custom pcap library, specifically the code by Phil Woods (Nice job by the way Phil). If you have not built snort against Phil’s libpcap that supports pcap ring buffers in shared memory, PCAP_FRAMES means absolutely NOTHING to you, zip, nout, fsck all, nada.

If you are unsure if you have built Snort against Phil’s libpcap or a stock distribution, then trust me you’re using a stock libpcap.

For those who are interested, PCAP_FRAMES defines a size (in frames) of a pcap ring-buffer in shared memory.

Are you sure? It looks like Snort stops with this as an error.

Yes I’m sure, and I find your lack of faith disturbing. Lets look at the code in snort.c to check.

1163     if( getenv(“PCAP_FRAMES”) )
1164     {
1165         LogMessage(“Using PCAP_FRAMES = %s\n”, getenv(“PCAP_FRAMES”) );
1166     }
1167     else
1168     {
1169         LogMessage(“Not Using PCAP_FRAMES\n” );
1170     }

If the environment variable PCAP_FRAMES is set, it shows the value to user during Snort initialization. If the environment variable is not set, it tell the user that PCAP_FRAMES are not being used.

For example, ill start up snort as a sniffer on my Mac with a stock libpcap.

[09:12:32]lward@drax~$ sudo snort -vdei en0 > /dev/null
Password:
Running in packet dump mode
-snip verbose startup output-
,,_     -*> Snort! <*-
o"  )~   Version 2.8.0.2 (Build 75)
''''    By Martin Roesch & The Snort Team:
http://www.snort.org/team.html
(C) Copyright 1998-2007 Sourcefire Inc., et al.
Using PCRE version: 7.6 2008-01-28
Not Using PCAP_FRAMES
^C*** Caught Int-Signal
==============================================================
Packet Wire Totals:

-SNIP-

Here Snort has started up and was sniffing without error (until I hit CRTL+C), now lets set PCAP_FRAMES to some garbage because it will have no effect on Snorts behavior or my stock libpcap.

bash-3.2# export PCAP_FRAMES="Foo Bar This setting has no impact on my libpcap instance"
bash-3.2# snort -dvei en0 > /dev/null
Running in packet dump mode
--== Initializing Snort ==--
Initializing Output Plugins!
Verifying Preprocessor Configurations!
Initializing Network Interface en0
OpenPcap() device en0 network lookup:
en0: no IPv4 address assigned
Decoding Ethernet on interface en0
--== Initialization Complete ==--
 ,,_     -*> Snort! <*-
o"  )~   Version 2.8.0.2 (Build 75)
 ''''    By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2007 Sourcefire Inc., et al.
Using PCRE version: 7.6 2008-01-28
Using PCAP_FRAMES = Foo Bar This setting has no impact on my libpcap instance
 ^C*** Caught Int-Signal
 ===================================

So in summary, this verbose message has no meaning to most users of Snort. If you are running Snort as an IDS but not in daemon mode, don’t expect to see anything on STDOUT until you stop the processes (hit CRTL+C to send a SIGINT).

As always, happy Snortin’
-Leon

Written by leonward

July 18, 2008 at 6:13 pm

Posted in Security

Tagged with

Defining achievable IDS/IPS deployment goals

leave a comment »

A network intrusion detection (and prevention) system is a flexible tool that can be used in many different ways. Let’s outline some of the most common deployment types I see in use today on *real* networks, and look them in no particular order. The reason for looking at these deployment type is to encourage more common compartmentalization (or segmentation) of monitoring tasks.

Firstly let’s don’t not forget that I[DP]S is all about access controls, which controls are implemented are your choice.

a) Tactical threat suppression
b) Business link risk mitigation
c) Security event detection
d) Network audit controls

Tactical threat suppression (Provides a preventative access control)
This is normally seen as the deployment of IPS at key access gateways of a protected network, the policy deployed is set to prevent specific malicious traffic flows from gaining entry. This design meets the “virtual patch” ideas to protect assets from key threats that concern the security team. Think “sploit de jour”.

Security event detection (Provides a detective access control)
Deployment of an IDS to detect network events that could impact the traditional security goals of the network (think network security 101 goals here (C, I & A)).
This is probably the most commonly planned IDS deployment from the out-set, it defines a system that is inspects network data flow, and when a security event occurs a team of analysts is there do their “job”. Following analysis, some form of incident response policy would be followed that should lead to the event being resolved. The main requirement for this type operation is a tuned IDS system to detect events that matter to the organization where something can be done in response to them.

Business link risk mitigation (Provides a preventative access control)
The use of an IPS can decrease the risk associated with a network link, therefore allowing the organization to potentially conduct business with higher risk 3rd party networks. The IPS policy acts as a traffic scrubber to prevent potentially harmful flows from entering the network from less-trusted parties.

Network event recording (Provides an audit control)
Deployment of an IDS that monitors the network for potential security events and supporting information. This is sometimes seen as a failed “Security Event Detection” deployment, where an IDS just logs event data but isn’t inspected by an analyst in anything close to real-time. A report may be run once in a while, but the data is stored for future reference should it be needed.
I see this is as a very valid deployment goal, and those who want “all rules enabled” generally fit into this category.

Problems can appear when designs attempt mix requirements between these achievable goals, for example:

Security event detection + Network event recording:

This combination leads to access and audit controls being enabled in the same policy. Those who are interested in audit requirements commonly want “all rules enabled” and therefore create an un-tunable policy that cannot hope to provide accurate security event detection (read bucket loads of F+).

The methods used to analyze, store, and work with event data may vary across the deployment goals. For example, if a user wants to place a device outside of a firewall to provide audit records, keeping event data in a live event analysis system may be overkill. Maybe a better solution would be an event feed to a SAN in a flat-file system. This would remove the burden to keep event data in an analysis database for real-time access.

Splitting an IDS/IPS deployment into logical chunks, each with specific requirements makes makes a far more manageable and valuable deployment as these goals can be segmented and managed on their own. When I get time I will put more effort into explaining my ideas around this, but in the short term I wanted to throw some ideas out there.

Written by leonward

July 8, 2008 at 6:12 pm

Posted in Security

Tagged with ,