An alchemists view from the bar

Network Security Alchemy

Posts Tagged ‘tuning

Automated IDS alert thresholding

leave a comment »

A question was posted to the Snort forums a short while back asking if Snort had the ability to accept how “dirty” a network is today, and then be alerted to only new threats that have never been seen before.

The idea behind this is quite simple, make an assumption that the network is “secure” as it is and operating fine today, then work out how much IPS noise it generates under this normal operation. In the future only alarm on events that have not been previously discovered during the “normal” period.  This type of difference analysis commonly requires a reporting tool that analyses the output from network sensors, but the way that the question was phrased got me thinking about this as a tuning process.

So lets summarize the steps involved to achieve this as a tuning goal.

  • Sensor must be deployed into the target network
  • Sensor must be run for a period of time to generate an alert “base line”
  • The Alerts must be investigated to check that they are all “acceptable” and normal for a network of this type (make sure you don’t accept an already p0wned network)
  • All alerts that were raised then need to be suppressed based on the assumption that they have no interest to the analyst now or in the future.
  • Sensor must be restarted with a new “suppressed” configuration.

It turns out that this is a simple thing to achieve, and after thinking about it for a while also raises two very interesting points:

  • The bulk trade off between F+ and F- . Is there an acceptable ratio, and what is it?
  • IDS for forensic use vs IDS for enhancing current security

I will approach both of the above points in another blog entry when I get some time.

Anyway, here is a dirty bit of perl that achieves what the above. It parses Snort’s fast alert output, and creates a suppression entry for each gid:sid that has generated an alarm. The suppression decision is made either on the event source IP, destination IP, or both IPs depending on what type of event is discovered.

For example, it is more common that policy-violations will need to be suppressed on the IP address that was the source of the event, and therefore the source of the “non policy violating policy-violation” that you don’t want to know about in the future (there is logic there, you just need to look for it).

Execution of the script is simple, and instructions are included in the source.

Download here.


Written by leonward

June 6, 2008 at 6:07 pm

Posted in Security

Tagged with ,

Leon’s tips for Ips event classification and analysis

leave a comment »

… and more tuning ideas

So here I am, stuck in another hotel bar, this time at the dark-end of Cornwall, sipping (carefully I may add) at the local tipple (Skinners Press gang cider). I have been taking some time to sift through snort forum postings and following the discovery of the normal “what the hell do I do about this alarm” calls for help, I thought I would take time-out to compile a list of top-tips to  better deal with intrusion alarms.

We all know the situation, you are being alerted to new “stuff” happening on the network, you don’t recognize the alarm as obviously bad, or even obviously dismissible and are desperately trying to work out how much, or even if you should be concerned.

There are certain things you can do to enhance the process of dealing with and understanding new alarms you haven’t any experience in, and due to the shifting threat spectrum (/me slaps self around the face for extreme buzzword abuse, and to mitigate the effect of the cider) new alarms arrive all the time.

Without wanting to write a book on incident analysis, ill split this over a couple of blog postings as some of this advice is conceptual and some introduces methods of approaching the problem, and other tips are tool specific.

First, the bleeding obvious (if you don’t do this already, pick up your security hat and coat and head for the door).

Understand the alarm.

This is such an important point that I must state it even though it sounds obvious. What does the event message really represent? Be sure to read all documentation associated with the alarm. If your IDS is Snort, and used the VRT rules, you have luck on your side as Nigel does a good job of writing and keeping rule documentation current. It is essential that you understand what the writer of the rule wanted to bring to your attention. Your reaction should vary depending on what has occurred, and one hopes that they would have attached a realistic event message. If you consider a regular server-side attack, all of the following may be raised in separate alarms associated with a single flow.

Access, or discovery of a potentially vulnerable system on the protected network
Exploitation of a vulnerability
Discovery of a specific exploit
Malformed network packets
Successful attack response

One of the things that you should have in your favor (and unfortunately, or rather shockingly, this isn’t always the case with many commercial tools) is the ability to look at the detection “source”. This provides an undeniable method of knowing what was being searched for, and may yield more clues to the writers intent (if not already obvious).

Understand the source and target devices in the context of the alarm.
Is it an attack response coming from your network? or someone else’s? How this is evaluated is specific to the event.

Take a look at the packet(s) that raised the alarm for supporting evidence of suspicion.

Without access to the packet that triggered the event(s) you’re dead in the water here and you are flying blind. Was this alarm real or a “programming error” on behalf of the detection system. Take a look at the decoded packet, inspect it, check for the presence of supporting evidence that the alarm may be real. A NOOP sled or a shell call are trivial examples.

Understand the quality of the rule/signature that generated the event.
Check out how the rule was written, if your IDS doesn’t provide you with the detection source, you are dead in the water. Being able to justify WHY an event triggered a rule is imperative.

The slightly less than obvious … the “C” word.

Understand the alert in the context of the systems involved.
To qualify the potential impact of an “attack” against a target system we need to know what the target system is. In turns out that this can be a harder job than discovering the attack itself, evaluate the alarm against the Operating System, Services, and applications provided by the target.

Evaluate the event in the context of the organization.
What was the “application” being attacked. By application I refer the the purpose of the target system in the operation of the organization. IIS alone means nothing, what is being served over HTTP from this device, and how does it’s compromise effect the operation of the company.

Following the completion of this process TUNE! Can you automate this effort the next time this event is raised on …
This target
A group of targets
Your whole network

Written by leonward

May 2, 2008 at 6:01 pm

Posted in Security

Tagged with ,

The natural habitat of a network security appliance

leave a comment »

…Challenging the black art of tuning.

Living things have a natural habitat, an environment where they thrive because it provides all that’s needed for their survival. The evolution, and more importantly extinction of species has shown us that once the habitat of a creature changes substantially, it must either adapt to its new surroundings or it can quickly become extinct.

The natural habitat of a network security device, such as a firewall or an intrusion prevention system is the modern enterprise network. From the outside this habitat looks to be volatile and hazardous where only the strongest technologies survive.

When an organisation is seeks to adopt a new security technology, they commonly go through a process of product selection based on a key criteria, for example.

Does the product meet business goals assigned to the project
Cost of ownership & return on investment
How well does the product integrate with the operating environment
How will it perform its function on our network rather than someone else’s

Integration questions and tests are particularly important as no two enterprise networks resemble each other, however one point that commonly gets overlooked is that the modern network rarely resembles itself a few months further along the line. How well security technologies deal with this rapid rate of change can be linked to how successful their deployment will be a few months along the line.  Will the new device adapt as the environment changes? Alternatively, will it continue in a pointless attempt to enforce extinct policies that has no relevance to the state of the organisation as it is now.

When introduced to a new network, an Intrusion Prevention System needs to be configured for the environment, this is so the device can better understand the habitat it operates in and is therefore better equipped to detect or prevent intrusions. In the world of IPS this is known as the black art of tuning. A tuning process can be broken down into a couple of logical steps.
Deploying vulnerability based network attack detection or prevention capabilities for assets that require protection.
Mapping the organisations acceptable usage policy into the devices configuration.

Both of these steps provide their own challenges, for the initial configuration of a system and also it’s adaption as the network it protects evolves. Lets take a look at each one in turn and discuss methods that can employed to improve the accuracy of detection, speed of response, and adaption to the network as it and associated business goals change.

Vulnerability based attack detection and prevention.

IPS is commonly considered the current generation of network intrusion detection systems, the new kid on the block that has the ability to prevent the exploitation of network vulnerabilities or violations of acceptable use as well as alert to the presence of an attack. Deciding on what vulnerabilities the device needs to detect or protect from exploitation has traditionally been based on user input. It is assumed that the security or network team within an organization is aware of the assets and services offered by the network, and therefore in a position to decide what vulnerabilities the IPS should mitigate. Unfortunately I commonly find this assumption to be flawed.

Many organizations I speak to incorrectly believe that they have a unique problem. Not knowing what assets and services are operating on the network, and therefore not knowing what needs to be protected for which vulnerabilities. This is clearly not a unique issue as I run into it all the time, and the impact of this problem turns out to be high. Missed attacks or false alarms.

Here at Sourcefire we designed a technology back in 2003 that provides information to make this task much easier, we call it RNA – Real-time Network Awareness. RNA provides a map of what the protected network looks like right now, based on how assets and services behave or are accessed. This real-time network map provides good answers to key questions before and after security events occur.

Before an event.
What devices are currently on the protected network?
What services do these devices offer?
What vulnerabilities may exist on these systems?
What detection or prevention capabilities do I need to employ to best protect this network?

After an event.
Was the attack relevant to the device or service?
E.g. Would the target have been vulnerable to the attack. Was it an Apache attack against an IIS Webserver.
Following the attack, did the network or asset change in any way?  2E.g. Did a new service start, or a new client application communicate to the internet for the first time.

Having this real-time map of assets on the network allows us to quickly adapt to changes in the environment. For example, take the current running IPS policy, maybe one that is designed to protect public facing assets from known attacks and overlay it onto a map of what the network is right now. Are there any gaps in the defenses? Has a new service been deployed on the network without the security team being made aware of it?
Has the version of Apache been updated on our production systems? Therefore mitigating the risk of some attacks being successful.

This real-time, constantly adapting map of the network is the key component of enabling the IPS to evolve as its habitat changes. It prevents it from becoming a dinosaur and churning out useless extinct log data.

Monitoring and Enforcing an Acceptable Use

Detecting violations of the organizations acceptable usage policy at a network level is a commonly desired function of Intrusion Prevention. Although we instinctively think of Firewalls, IPS and other network access control devices preventing communications between specific computers and protocols, these communications most likely occur at the request of a user. A laptop for example is not intrinsically a malicious device as its functions are controlled by a user, so if an acceptable usage policy is violated through an instant messaging chat session, do we blame the device or the user?

Tracking down the sources of AUP violations can traditionally be tricky in dynamic environments, and as it happens these are the most common source of AUP violations. Large DHCP ranges are commonly associated with call centres or groups of office staff who will gladly whittle their work day away by abusing network resources. In these environments it can be hard to find something static to associate with an event to allow investigation. The IP address of the source has since changed, knowledge of the original MAC address has been lost due to the network topology, a user was “hot-desking”, the only static in this habitat is the person that violated policy.  This is why it is important to associate these types of events with the user of the system at the time of the violation.

This unique combination of user and network awareness providing an up-to-date map of who is accessing what on the network is invaluable when it comes to actually enhancing the security. Network information has its most value at the time of discovery, constant discovery means providing constant value.

Written by leonward

March 14, 2008 at 6:00 pm

Posted in Security

Tagged with , ,