An alchemists view from the bar

Network Security Alchemy

Archive for the ‘Sourcefire’ Category

Pushing the OpenFPC project forward

leave a comment »

A couple of people have been working harder than normal over the last couple of weeks. Edward, and I are happy to push out another OpenFPC test release to the world.

Here is short list of highlights and changes, however there is one point to pay close attention to.

A very kind web developer has started to help the team work on a central user interface for searching and extraction. Ill introduce him and his work in another future post, however in the short term thanks should be sent over to Eduardo!

0.3 Change highlights

  • Multiple configs can co-exist on a single box
  • Sourcefire IPS event parsing fixed
  • Snort-Fast event type no longer required port numbers. Makes multi-session extracts more simple (http attacks for example)
  • Search via bpf (–bpf command line option to openfpc-client)
  • Passwords no longer echo to screen
  • New init scripts to work with the new openfpc command
  • LSB compliant init scripts
  • Better log output (wlog) and verbose message handeling
  • Added better example configs (openfpc-default.conf and openfpc-example-proxy.conf)
  • Enabling session data is now far more simple
  • Included web-ui, now enabled by default
  • Space now renders in GB rather than Bytes
  • Fixed performance hit on cx2db inserting half open sessions.
  • Improved help text
  • The out-of-the-box proxy and node configurations now work with each other
  • CGI interface for full packet integration with other tools

As always, feedback and bugs are welcomed.



Written by leonward

November 22, 2010 at 9:09 pm

Technology Evaluation Do’s and Don’ts

with one comment

As an engineer that has been involved with many IPS evaluations, I have first hand experience of the deciding factors that make the difference between an optimal test, and a test process that becomes drawn out, consumes far too much resource, and provides non confident results back to the business. It is my opinion that a successful evaluation should result in a repeatable and deterministic test report that can be used for the final justification for the selection, or non-selection of a product.

In order to provide some advice of how to optimize the evaluation process without influencing its result or a customers test criteria, I have created a list of “Dos and Don’ts” that can be considered during the planning and testing process. Following these points will avoid many common pitfalls I have witnessed first-hand and should result in a much better test experience regardless of vendor or technology.

This list was created based on my Intrusion Prevention, Vulnerability Assessment, and network discovery background, however the concepts I believe should hold true for most other technologies. I have purposely avoided sharing specific test methods as they do (and should) vary as the customers needs vary. Contact me if you would like some help with those as well.

I hope the below list is if use, and improves your evaluation experience regardless of the technology or vendors involved.

  1. Do: Understand the business needs for implementation and keep them the top priority during test plan creation.
  2. Do: Create a list of functional requirements ahead of the testing process.
  3. Do: Create a list of how each test should be performed (method).
  4. Do: Have the test list reviewed by peers and vendors involved. They may catch impossible, irrelevant, or incorrect scenarios.
  5. Do: Work through the test list assessing a compliant/not-compliant state next to each test with comments.
  6. Don’t: Waste time testing a function of a device that the vender states is not supported/included/doesn’t work.
  7. Don’t: Change the test criteria during the testing.
  8. Don’t: Enforce how (method) a capability should be achieved, assess that it can be achieved (result).
  9. Do: Score and provide weighting to each test. Not all test criteria are, or should be, of equal in importance
  10. Do: Make use of vender/reseller/consultant expertise during a test. An evaluation of technology should evaluate solely the technology not the evaluator’s knowledge of how to use that equipment.
  11. Do: Test the support and after sales capability of the vender/reseller chain. This will be your primary “help” contacts if selected, make sure they can help you!
  12. Do: Check the DUT (Device under test) can integrate with all systems/services that will be in use in the product environment.
  13. Do: Test unique capabilities (USPs) of the DUT, if they are relevant to the functional requirements.
  14. Don’t: Waste time on irrelevant unique capabilities (USP) that are not part of your functional requirements.
  15. Don’t: Try to “Level the playing field” by only comparing capabilities offered by both devices in order to achieve an “apples-to-apples” test. Focus on the business requirements.
  16. Do: Remember that no two systems are the same; there will be differences in both function and implementation. Don’t fear change from any system you are used to and assume something different is something wrong.
  17. Don’t: Assume that multiple products from a single vendor work well together and integrate, either now or in the future. Test them to get your own opinion of the integration maturity.

If you have any more to share from your own experience, I’m open to listening and learning from others.


Written by leonward

July 22, 2010 at 9:29 am

Posted in Security, Sourcefire

OpenFPC 0.1a – Installation and usage

with 2 comments

Firstly, I need to vent my anger at WordPress’s post formatting breakage. It is impossible for me to format this post using the Visual editor, and the HTML generated is so unbelievably ugly I can’t follow it.  So when the formatting of the text below doesn’t look correct blame wordpress not me.

I have uploaded a tarball of OpenFPC-0.1a to Googlecode, and as there isn’t any real documentation yet for OpenFPC I thought I would provide a few tips for people who want to try it out.

Firstly a word of warning: OpenFPC could be changing in the near future. I’m meeting up with ebf0 the author of FPCGui in June for a beer. Because duplicating effort is never a smart thing to do, and as both projects have similar goals it may make sense to pool some resources. Anyway, for the people who emailed me after my last post wanting to know how to get started with OpenFPC here are some tips.

The below is being performed on Ubuntu 10.04 LTS.

1) Check for requirements

OpenFPC depends on a few libraries and tools that need to be installed on the server you decide to dedicate for traffic collection. These include:

  • TCPdump
  • tshark (Part of the wireshark project)
  • mergecap (Part of the wireshark project)
  • daemonlogger (Version 1.2.1 or greater)
  • Perl
  • Perl Getopt::Long

On Ubuntu 10.04 LTS, all of these applications and librariers can be installed via apt without any real challenges. If they are not available in your Operating System’s package manager, you will have to go download and install them yourself.

lward@UbuntuDesktop:~$ sudo apt-get install tcpdump tshark daemonlogger

2) Install OpenFPC

Once those packages are installed, go download OpenFPC from here. This document was written for version 0.1a, but if there is a later release on the download list it’s best to use that. Once you have the tarball, extract it to somewhere in your home directory.

lward@UbuntuDesktop:~$ tar -zxvf openfpc-0.1a.tgz

If your dependencies are all satisfied, installation should be as simple as running the installer. And that should be it!

OpenFPC installs itself into /opt/openfpc, and looks for a configuration file in /openfpc.conf /etc/openfpc/openfpc.conf, /opt/openfpc/openfpc.conf (first config file found wins).
I suggest you start off by editing the file /opt/openfpc/openfpc.conf. Pay close attention to the FILE_SIZE and DISK_SPACE values. You will want to increase the FILE_SIZE value to over 10M for a production environment, it’s that default because of my testing on a VM. 1G or 2G would probably make more sense.
DISK_SPACE equates to a percentage of disk space to use on the capture partition. All other values should be pretty much obvious.
Once you’re happy with the values you have set, start up openfpc.

TIP: Because openfpc looks for a .conf file in your current working directory, be careful where you start/stop/extract pcap files from. I think I may change this behaviour in the future, but this is how it is for now.

lward@UbuntuDesktop:~$ /etc/init.d/openfpc status
[*] Reading configuration file /opt/openfpc/openfpc.conf
[!] No current buffers found in /var/tmp/openfpc – Have you started it yet?

lward@UbuntuDesktop:~$ sudo /etc/init.d/openfpc start
[sudo] password for lward:
[*] Reading configuration file /opt/openfpc/openfpc.conf
[-] Daemon mode set
[-] Interface set to eth0
[-] Logpath set to /var/tmp/openfpc
[-] Log filename set to “openfpc-pcap”
[-] Pidfile configured to “openfpc-dl”
[-] Pidpath configured to “/var/run”
[-] Rollover configured for 10 megabytes
[-] Rollover configured for 0 none
[-] Pruning behavior set to oldest IN DIRECTORY

DaemonLogger Version 1.2.1
By Martin Roesch
(C) Copyright 2006-2007 Sourcefire Inc., All rights reserved
Checking partition stats for log directory “/var/tmp/openfpc/.”
50% max disk utilization = 2467159 blocks free (out of 4934317)
 Blocksize = 4096
Rollsize = 2560 blocks
[-] It looks like daemonlogger has started successfully
[*] Traffic buffer (Daemonlogger) started on Tue May 25 17:16:28 BST 2010

Currently pcaps need to be requested locally using the tool, and only Snort-syslog, Snort alert_fast, Sourcefire3D, and Exim4 log entries are supported. If you would like support for another log format added that you use right now, send me samples via email (<myfirstname> Please only ask for log file formats that you are in a position to use and test, don’t just suggest nice-to-have.

So once you have a log entry you want to extract, run the command. I have one of my Snort instances writing events to syslog. I don’t make use of a nice Snort event UI like Snorby, Sguil, or Base on this device but when an event is triggered I want some more packet data.Here is an alert Ill use as an example (IP address’ are censored of course):

May 26 08:36:45 XXXXX snort: [1:2020:5] RPC mountd TCP unmount request [Classification: Attempted Information Leak] [Priority: 2]: {TCP} X.X.X.X:991 -> Y.Y.Y.Y:41208

To extract the session based on this log message, I will pass the -a “<LOG LINE>” arguments to

leon@rancid:~$ -a “May 26 08:36:45 rancid snort: [1:2020:5] RPC mountd TCP unmount request [Classification: Attempted Information Leak] [Priority: 2]: {TCP} ->”
*  – Part of the OpenFPC Project *
Leon Ward –
– Searching for traffic..
– Merging …
– Created /tmp/extracted-1274864016.pcap (1.1K Bytes)

I now have the complete session in the file /tmp/extracted-1274864016.pcap.
If you want to create your own wrapper scripts around ofpc-extract, you could use the -q (quiet) flag e.g.

leon@rancid:~$ FILENAME=$( -a “May 26 08:36:45 rancid snort: [1:2020:5] RPC mountd TCP unmount request [Classification: Attempted Information Leak] [Priority: 2]: {TCP} ->” -q)
*  – Part of the OpenFPC Project *
Leon Ward –
leon@rancid:~$ echo $FILENAME

Good luck with your sniffing!

Written by leonward

May 26, 2010 at 10:15 am

Introducing OpenFPC – An Open Full Packet Capture Framework for Network Traffic Recording

with 6 comments

I’m writing this post in order to introduce a project I’ve been thinking about for years and started working on a couple of months back. OpenFPC.


Update: Aug/2010: OpenFPC now has a new home: Go there for more info.

OpenFPC is designed as an add-on capability for other network communication and security technologies, it provides those technologies (open source or comercial) with something that their users often demand but the system doesn’t or can’t deliver on. The complete network traffic associated with a network security event. OpenFPC provides a central interface where full traffic session data can be automatically (or manually) requested from a distributed recording infrastructure, and then deliveries it back to the requester in a .pcap file or as a href to the .pcap file.

So what makes,or will make OpenFPC different from other tools available that provide similar functionality?

  • Distributed framework – With a central user interface and request point.
  • Quick and simple to set-up and start functioning. No expert knowledge required
  • Automated extraction of sessions (Syn-to-fin) of known to be “interesting” events for longer-term storage
  • Traffic is requested in the context if the requesting device, system or process
  • No database requirement for traffic/session data indexes
  • Optimised session searching over large traffic archives
  • Web UI, command line, and automated-process interfaces all provided

The above points are in no particular order, but to introduce the project I would like to focus on one feature in particular. In further posts and as the code develops I plan to expand on many of the above with functioning examples.

“Traffic is requested in the context if the requesting device, system or process”

A wise man once said that a picture is worth a thousand words, maybe the below examples will provide the same result and save me a load of typing.

Example 1) Mail log context

Problem: My mail log shows an entry that I have further interest in, this transmission is suspected to contain some malware. I would like to inspect the complete network traffic session to get hold of the malicious stuff if it actually exists.

leon@rancid:~$ tail -n 1 /var/log/exim4/mainlog

2010-05-16 11:35:27 1ODbBv-0008UQ-20 <= [] P=smtp S=1019

Solution: Extract the session based on the Exim4 (my mail daemon) log line. Below is the log entry that I’m interested in.

leon@rancid:~$ -a 22010-05-16 11:35:27 1ODbBv-0008UQ-20 <= [] P=smtp S=1019
*  – Part of the OpenFPC Project *
Leon Ward –
– Searching for traffic…..
– Merging …
– Created /tmp/extracted-1274006391.pcap (3.4K Bytes)
Now I have the complete session to inspect and perform analysis on.
Example 2) IPS Context
Problem: Looking through my central log manager I spot the following IPS event that was passed to my log manager over Syslog. Because the event was uploaded to my screen via syslog i’ve lost all packet data.
May 16 11:49:05 rancid snort: [1:1951:7] RPC mountd TCP mount request [Classification: Attempted Information Leak] [Priority: 2]: {TCP} ->
I would like to extract the complete session to perform further investigation into the event.
leon@rancid:~$ -a “May 16 11:49:05 rancid snort: [1:1951:7] RPC mountd TCP mount request [Classification: Attempted Information Leak] [Priority: 2]: {TCP} ->
*  – Part of the OpenFPC Project *
Leon Ward –
– Searching for traffic.
– Merging …
– Created /tmp/extracted-1274006982.pcap (1.2K Bytes)
Example 3) Arbitrary extraction
Of course you don’t need to format a request like  a log file message for a session, simply search. Here are the command line options.
leon@rancid:~$ --help
*  - Part of the OpenFPC Project *
  Leon Ward -
- Usage:
  --mode     or -m <at|window> At a specific time, or search in a window
  --src-addr or -s           Source IP
  --dst-addr or -d           Destination IP
  --src-port or -u           Source Port
  --dst-port or -r           Destination Port
  --write    or -w           Output file
  --http     or -l                      Output in HTML for download
  --verbose  or -v                      Verbose output
  --debug                               Debug output
  --quiet                               Return only a filename or an error
  --all                                 Check in all buffers not just current sniff buffer
  ***** Operation Mode Specific Stuff *****
  "At" mode.
  --each-way or -e         Number of pcaps each-way Default: 1
  --event    or -a         Parse a supported event log line e.g. Snort, Sourcefire, Exim etc
  --timestamp or -t                     Event mode - Look each way of this epoch value
  --sf                                  Timestamp in SF format, convert it
 "Window" mode.
  --start    or -b                      Start timestamp for searching in absolute mode
  --end      or -j                      End timestamp for searching is absolute mode
If you are still reading this post, I assume that this project is something that may interest you and can guess you have a couple of questions:
  • What log formats are currently supported?
    Right now not many, this is a preview release after all. Snort Fast alert, Snort Syslog, Exim4, and Sourcefire 3D Defense Center, Sourcefire 3D Sensor.
  • When can I expect the central manager / WebUI
    Don’t hold your breath right now 🙂 But seriously, I’m working on it and had some ghetto code working that I’m too shamed of to share.
  • What’s next? When can I expect something stable?
    This is the most important question. I’m aiming for a stable non-distributed release without the manager in the next couple of weeks. I think it’s best to get a small but stable feature-set into the community rather than one big buggy release in a couple of months time.
  • Can I help?
    What a wonderful question!? Yes you can! Right now I need to find problems and iron out some bugs. In a couple of weeks Ill be looking for someone who can write a simple web UI for me, my un-themed s don’t look great
  • How do I install, configure and get this preview release running?
    Keep your eyes open for a blog posting in a couple of days talking you through an install on Ubuntu
  • What are the requirements to run? Do you use sancp or anything like that?
    daemonlogger (Thanks Marty!), tcpdump, perl
  • When’s the automated extraction bit coming?
    I’m not sure if this will be before or after the webUI. I have a day job after all!
  • Will you support <foo> log file format?
    Maybe, I want to have built-in support for most common log formats, tell me what you need and send samples.
  • Where can I get the code from?
    Stay tuned for an alpha release this week on

Happy Sniffing!

Written by leonward

May 17, 2010 at 8:00 am

Planing for infosecurity 2010 (London) – VRT Workshop

leave a comment »

It’s that time of year again. Infosecurity Europe is upon us. If you’re going to be there and have a strong interest in the inner-works of intrusion prevention engines, have I got a treat for you lucky lucky people 🙂

Sourceifre’s own Lurene Grenier (@pusscat) and Matthew Olney (@kpyke) are running a workshop!

Here’s the official blurb

Sourcefire VRT Workshop
Register here
– 11:00 – 12:00, Wednesday 28th April 2010 –  Mayfair Room, Earls Court.
The VRT team will demonstrate the power and flexibility of the engine by unveiling a new multi-faceted, scalable detection methodology targeted at addressing the most difficult detection problems facing security professionals today.

So if that gets your interest going, make sure you register here.

As a bonus for you all, Sourcefire are also running an “Intelligent Network Security” Workshop.

Sourcefire Intelligent Network Security Workshop
Register here | 15:00 – 16:00am, Wednesday 28th April 2010 | Mayfair Room, Earls Court
In the age of the advanced persistent threat of cloud computing and of new economic realities, how can companies ensure their networks are monitored and protected securely and cost-effectively? Find out how Sourcefire, the leader in context-aware Intrusion Prevention Systems has addressed the limitations of current generation IPS to provide true intelligent network security solutions.

And I’ll be stuck on the Sourcefire stand for most of the three days, pop buy if you want to say “oh hai”.


Written by leonward

April 1, 2010 at 9:13 am

Posted in Sourcefire

Tagged with , ,

Sourcefire Quick Tip: Custom RNA Service Detection

with 2 comments

As part of my Sourcefire “Quick Tip” series, a new video has been uploaded by our marketing guru’s to YouTube.

This time I ramble on about creating custom service detectors for Sourcefire’s RNA engine. This allows you to detect the network applications  in use on your network and track them in the context of IPS events, real-time network change detection, Service and host white-listing as well as general network mapping.

I hope it’s of use to you out there. I haven’t got my next quick-tip subject planned yet, so if anyone has a suggestion please let me know.

RNA Custom Services

Adding RNA Custom Services

If you would like to learn more about RNA, take a read here.


Written by leonward

October 26, 2009 at 11:55 am

Posted in Sourcefire

Sourcefire Quick Tip: Host Input API and HippiEd

with one comment

The on-line marketing team over at Sourcefire’s world domination HQ asked me to throw together some content for our Facebook / Youtube pages.

The result is a series of video quick tips showing some rarely talked about features of Sourcefire’s products. The first video is now available on the Youtube channel and Facebook page. The tool mentioned is available here.



Expect a few more over the next couple of weeks.


Written by leonward

October 14, 2009 at 8:41 am

Posted in Security, Sourcefire