An alchemists view from the bar

Network Security Alchemy

Big OpenFPC release – 0.6

with 2 comments

Pushing forwards closer to a 1.0 release for OpenFPC, one of the major components has now been updated – The GUI.

To introduce this new release I’ve put together a short screen-cast of OpenFPC to show the installation, setup procedure, and a bit of general usage. So if you’re tasked with rolling together your own full packet capture/network traffic recorder/forensics system, perhaps you may want to take a look below.

 

For those who don’t want to sit through five minutes of video to see what the new GUI looks like, here are a few screenshots of the system in action.

Version 0.6 is now available at  http://code.google.com/p/openfpc/downloads/list . Expect a few bugs, and if you report them, Ill own the task of fixing them.

-Leon

Advertisements

Written by leonward

June 13, 2011 at 12:37 pm

Posted in OpenFPC, Security, snort, Uncategorized

Tagged with , , ,

A new look for OpenFPC – New GUI in devopment

with 2 comments

Developing open source software has its ups and downs. It’s great to hear that your work is helping others solve problems they have, but on the flip-side some people simply love to focus on negatives and never offer to help improve through collaboration.

A user of OpenFPC recently decided they didn’t like the web UI much, and rather than simply complaining about it, they decided to collaborate and work on an overhaul. It’s efforts of people like this that make OSS all the more rewarding.

The UI isn’t quite ready to be released in an installable form, but I thought I would provide a couple of screenshots to wet current users appetite. David, thanks for your effort!

Written by leonward

April 25, 2011 at 6:56 pm

Posted in OpenFPC, Security

Tagged with , ,

Immunet 3.0, ClamAV, and OpenFPC updates (including a blatant product plug)

with one comment

I’m always pretty careful to keep anything too commercial away from my blog, but from time to time something just has to give.

Back in late (very late in fact) 2010, Sourcefire (those nice people who supply me with beer-money) purchased an exciting company called Immunet. Ill spare you the purchase details,  because it’s out-of-scope for this quick update.

I’ve been aware of Immunet for quite some time but haven’t had a chance to really use their technology in anger because I’m a OSX/Linux user, but this changed a couple of weeks back. I recently needed to use a Windows XP VM to work with some win32 only software, I’ve had a virtual machine installed for ages and because it’s rarely used it’s rarely updated (bad Leon!). I probably spend less than an hour a year on this windows VM, I simply don’t have time to install updates because I only use it for quick tests (very bad Leon!).

Immunet’s cloud architecture is perfect for AV in this type of environment, I never need to update my signature pack because all detection is performed in the cloud. While trying to install some software from a USB key-fob that was shared around at a recent conference what popped up? Immunet kindly did it’s job and protected me from some malware nastiness. Now that was awesome.

Oh, by the way Immunet isn’t only awesome (because it saved me from my own stupidity), its also $free and uses Clam AV (that’s also free, but as in speech as well). If you’re using a Windows VM or real device without AV you know what you should do… Go install Immunet for free now http://www.immunet.com . Go on do it now!

For those of you who read this blog for updates on OpenFPC, if you have any spare time please test the updated 0.5 release. There have been many changes at the back-end that I would like to get some feedback on. If it stops working or fails to start please let me know via the usual routes. You shouldn’t see many functional changes, but was a big massive re-write under the covers.

-Leon

Written by leonward

February 18, 2011 at 3:48 pm

Posted in OpenFPC, Security

Tagged with , ,

Insta-Snorby 0.4 with OpenFPC

with 3 comments

Snorby had a big launch this weekend with an event that rivaled Apple in terms of hype and excitement! The two-dot-ooh-yeah release has reached the unwashed masses.

The Snorby 2.0 feature that I’m most excited about is the inclusion of support for OpenFPC directly in the Snorby UI (but face it I’m kind of biased here). Many users of Snorby will be unaware of the OpenFPC project, and as they could be eager to try out the bleeding Snorby version, I thought I would include a quick how-to (below) of adding OpenFPC on to the Insta-Snorby appliance.

OpenFPS and Snorby together

I wouldn’t expect real-world users of Snorby / OpenFPC to use the Insta-Snorby VM, but it’s a good introduction / test platform. As a guide to effort, the below ten steps should take about ten minutes to follow (including the download and updating of packages).

If you spot any errors please let me know, this is the bleeding edge after all.

First the obvious bits….

1) Download the Insta-Snorby-0.4.iso

2) Install the .iso on to the hard disk of a virtual (or physical) machine

3) SSH in to the device as root.

Now the less-obvious bits…

4) Prepare the platform.

Update the package archives. This is mandatory, it’s not being performed as part of good practice.

root@Insta-Snorby ~# apt-get update

Install the dependancies from the Ubuntu package archive (note you can copy/paste the below into your ssh session rather than re-type).

apt-get install apache2 daemonlogger tcpdump tshark libarchive-zip-perl \
libfilesys-df-perl libapache2-mod-php5 mysql-server php5-mysql \
libdatetime-perl libdbi-perl libdate-simple-perl php5-mysql \
libterm-readkey-perl libdate-simple-perl

5) Download the latest version of OpenFPC from http://code.google.com/p/openfpc/downloads/list

root@Insta-Snorby ~# wget http://openfpc.googlecode.com/files/openfpc-0.4-266.tgz

Note that 0.4-266 is “current” at the time of writing, but there is a lot of development happening right, so make sure you get the latest and don’t assume 0.4-266 is still “current”

6) Install OpenFPC

root@Insta-Snorby ~# tar -zxf openfpc-0.4-266.tgz 
root@Insta-Snorby ~# cd openfpc-0.4-266/
root@Insta-Snorby ~# ./openfpc-install.sh install

You will be promoted to provide a password for the OpenFPC extract.cgi script. This password protects any attempts to pull out a pcap via the cgi interface used by Snorby via Apache’s basic auth. It saves the password to /etc/openfpc/apache2.passwd.
You will need this username/pass to access any pcaps via Snorby, so REMEMBER IT!

7) Customize OpenFPC

OpenFPC is a client/server system, the openfpc-client does not need to be on the same physical host as the openfpc-queue daemon and therefore it listens on a network socket (default 4242). The default username and password is

Username: openfpc
Password: openfpc

If you want to change these, edit /etc/openfpc/openfpc-default.conf and set…

a) USER=openfpc=openfpc

Set this to whatever username/pass you desire e.g.
USER=snorby=letmein

b) Change the user account that is used to pull PCAP files via the extract.cgi interface to one you have specified with a USER definition. e.g. for the above user definition I would use:

GUIUSER=snorby
GUIUSER=letmein

8) Start up OpenFPC

root@Insta-Snorby ~/openfpc-0.4-266# openfpc –action start

###############################################################################
[*] OpenFPC instance openfpc-example-proxy.conf
 -  NODENAME:              Example_Proxy
 -  DESCRIPTION:           "An example OpenFPC Proxy config. www.openfpc.org"
 -  STATUS :               DISABLED 
 -  PORT:                  4243
###############################################################################
[*] OpenFPC instance openfpc-default.conf 
 -  NODENAME:              Default_Node 
 -  DESCRIPTION:           "An OpenFPC node. www.openfpc.org" 
 -  STATUS :               ENABLED
 -  PORT:                  4242
 -  INTERFACE:             eth0
 -  FULL PACKET CAPTURE:   ENABLED
 -  PACKET STORE:          /var/tmp/openfpc/pcap
 -  SESSION DATA SEARCH:   DISABLED
Starting Daemonlogger (Default_Node)...                                    Done
Starting OpenFPC Queue Daemon (Default_Node)...                            Done

9) Check communications and your openfpc username/password.

Use the command line tool openfpc-client to check things are working. The –action status will provide a status check of a remote OpenFPC instance.

root@Insta-Snorby ~/openfpc-0.4-266# openfpc-client -a status
   
 * openfpc-client 0.4 *
Part of the OpenFPC project
Username: openfpc
Password for user openfpc : 
#################################### 
 OpenFPC Node name   :  Default_Node 
 OpenFPC Node Type   :  NODE
 OpenFPC Version     :  0.4
 Oldest Packet       :  1291638906 (Mon Dec  6 12:35:06 2010)
 Oldest Session      :  0 (Thu Jan  1 00:00:00 1970)
 Packet utilization  :  10% 
 Session utilization :  Disabled% 
 Session DB Size     :  Disabled rows 
 Session lag         :  0 files 
 Storage utilization :  10% 
 Packet space used   :  1867896 (1.87 GB)
 Session space used  :  Disabled (Disabled Bytes)
 Storage used        :  1867896 (1.87 GB)
 Load avg 1          :  0.04 
 Load avg 5          :  0.05 
 Load avg 15         :  0.08 
 Errors              :  0 
root@Insta-Snorby ~/openfpc-0.4-266#

10) Configure the Snorby OpenFPC plugin

Navigate to the Snorby web interface, and browse to Administration.

Enable your OpenFPC integration here

  • Check the box “Enable OpenFPC support”
  • Use the below URL for extraction
  • Hit “Save Settings”

Complete!

Now when you look at an IPS event, you will have a “Packet Capture” button that pulls out the complete session data via OpenFPC.

Many of the advanced OpenFPC capabilities are not addressed in this how-to such as connection/flow capture and searching, compressed extracts, reports, distributed extracts, horizontal scaling, etc etc but I’m keeping this How-to simple. If you want to know more, you know where to look http://www.openfpc.org.

-Leon

Written by leonward

December 6, 2010 at 1:24 pm

Pushing the OpenFPC project forward

leave a comment »

A couple of people have been working harder than normal over the last couple of weeks. Edward, and I are happy to push out another OpenFPC test release to the world.

Here is short list of highlights and changes, however there is one point to pay close attention to.

A very kind web developer has started to help the team work on a central user interface for searching and extraction. Ill introduce him and his work in another future post, however in the short term thanks should be sent over to Eduardo!

0.3 Change highlights

  • Multiple configs can co-exist on a single box
  • Sourcefire IPS event parsing fixed
  • Snort-Fast event type no longer required port numbers. Makes multi-session extracts more simple (http attacks for example)
  • Search via bpf (–bpf command line option to openfpc-client)
  • Passwords no longer echo to screen
  • New init scripts to work with the new openfpc command
  • LSB compliant init scripts
  • Better log output (wlog) and verbose message handeling
  • Added better example configs (openfpc-default.conf and openfpc-example-proxy.conf)
  • Enabling session data is now far more simple
  • Included web-ui, now enabled by default
  • Space now renders in GB rather than Bytes
  • Fixed performance hit on cx2db inserting half open sessions.
  • Improved help text
  • The out-of-the-box proxy and node configurations now work with each other
  • CGI interface for full packet integration with other tools

As always, feedback and bugs are welcomed.

 

Written by leonward

November 22, 2010 at 9:09 pm

An OpenFPC Example: Clustering packet capture over multiple links/devices/countries.

with 3 comments

It’s been a while since my last post, but it’s because I’ve been busy working on ofpc. To rectify that, I thought I would share some of the concepts that are behind how OpenFPC should be able to grow rapidly into a distributed system.

One of the more useful features of ofpc is its self-referencing method for scaling out master/master/slave devices. This concept gets interest when I explain it to people, however it’s not really documented anywhere. So let me introduce it here with a working example……

There are a few common situations where the master/slave relationship can provide real value via clustering.

  • Geographically separated network links with guaranteed or possible asymmetric traffic paths
  • Multi-link trunks
  • High(er) speed links where you need to spread traffic load over multiple slaves

Firstly, please forgive my terrible retro-diagram skills.

OpenFPC Cluster diagram

So here’s the situation:

There are two pipes between network “A” and network “B”, and for whatever the reason, you don’t know if the traffic you want to grab from the buffer could be in the archive of SLAVE1 or SLAVE2. You do know however it’s going to be in one or more of them. Combined they become one *logical* network link.

By requesting the data from the Master queue daemon responsible for these two devices (MASTER in the diagram here), without specifying which slave you want to route your request to, it will search/extract from all of the slaves below it. The master ofpc-queued doesn’t need to be on a separate bit of hardware, it’s just represented in the diagram that way.

Here’s an example of it functioning in my test environment.

lward@UbuntuDesktop:~/code/openfpc$ ./ofpc-client.pl  -a fetch \
 --src-addr=192.168.222.1 --dst-port=22
* ofpc-client.pl 0.1 *
Part of the OpenFPC project
Username: master
Password for user master : 
#####################################
Filename: /tmp/extracted-ofpc-1284615954.pcap
Size    : 7.0M
MD5     : a495c1f38dce3dc9dff50ead47a415ab
lward@UbuntuDesktop:~/code/openfpc$

 

This ofpc request provided me with a 7MB pcap file made up from the traffic seen by “slave1” and “slave2”, it’s all merged together so I can inspect the traffic as the logical link processes it rather than what can be captured on one physical leg of the link. This isn’t limited to a maximum of two slaves, it can of course be many many more.

If for any given reason I would still prefer to only look at the traffic on one slave, I can either:

  • Make an ofpc request directly to one of the ofpc-slave devices
  • Specify the device to focus on to the master

For example…..

lward@UbuntuDesktop:~/code/openfpc$ ./ofpc-client.pl  -a fetch \
--src-addr=192.168.222.1 --dst-port=22 -o 4240 --device slave2
* ofpc-client.pl 0.1 *
Part of the OpenFPC project
Username: master
Password for user master :
#####################################
Filename: /tmp/extracted-ofpc-1284616271.pcap
Size    : 6.0M
MD5     : 68132e2e12c16665913cb1e7f36336f3
lward@UbuntuDesktop:~/code/openfpc$ 

If you want to test this feature out, make sure you’re using the latest openfpc code out of svn.

-Leon

Written by leonward

September 24, 2010 at 12:42 pm

Posted in OpenFPC, Security

Tagged with , , ,

OpenFPC Test Release

leave a comment »

The weekend has landed, and I have time to pull together some of the bits I need for an OpenFPC (Open Full Packet Capture) release, but I need your help.

I know there are bugs that still need squishing (Master-mode install script for example), but if you have time and are interested, please help me test out an alpha release. Go grab it from here (download the latest version number, it may change repeatedly over the next few days) and run the installer.

So far, I have only tested it on Ubuntu 10.4, the Redhat auto-dependency checking isn’t there yet but it should work on that platform if you have the required RPMs installed with a little tweaking.

So what are you waiting for!? Find problems, tell me where the install and setup falls down, and have some fun.

-Leon

Written by leonward

September 10, 2010 at 5:35 pm

Posted in OpenFPC, Security

Tagged with , ,