Blog View

0

eBPF and XDP support is one of the latest evolutions of the Suricata engine’s performance capabilities. These two technologies have been introduced recently for the Linux kernel and Suricata is one of the first well established and mature projects to make use of them. eBPF was introduced in the Linux kernel to be able to run user provided code safely inside the Linux kernel and XDP is running eBPF code on the network data path, as close as possible to the Network Interface Card.

The initial support for eBPF and XDP was initially available in Suricata’s 4.1, released in November 2018, and it has been greatly enhanced in Suricata 5.0. The development team at Stamus Networks, lead by Éric Leblond, has been the primary developer of eBPF and XDP support within Suricata. The latest additions, list of features and potential use cases enabled by eBPF and XDP are becoming more significant, even vital, making this an opportune time to provide a high level overview for the community. This is the purpose of this article written by Éric Leblond and Peter Manev.

Download the whitepaper: Introduction to eBPF and XDP support in Suricata

Stamus Networks

Stamus Networks has created a Network Traffic Analyzer (NTA), which uses network communications as a foundational data source for detecting security events, and married it with an Intrusion Detection System (IDS), which provides deep packet inspection of network traffic using a rules based engine. The combination of these two approaches, done simultaneously in a single solution, provides a level of correlated data never previously achieved. We then added a Threat Hunting interface to allow security practitioners the ability to quickly and efficiently search through this security data to examine, validate, and resolve the security incidents that they face on a daily basis.

Suricata

Suricata is a free and open source, mature, fast and robust network threat detection engine.
The Suricata engine is capable of real time intrusion detection (IDS), inline intrusion prevention (IPS), network security monitoring (NSM) and offline pcap processing.
Suricata inspects the network traffic using a powerful and extensive rules and signature language, and has powerful Lua scripting support for detection of complex threats.

Suricata’s fast paced community driven development focuses on security, usability and efficiency. The Suricata project and code is owned and supported by the Open Information Security Foundation (OISF), a non-profit foundation committed to ensuring Suricata’s development and sustained success as an open source project.

0

SELKS 5 is out! Thank you to the whole community for your help and feedback! Thank you to all the great Open Source projects and tools mentioned below for making it possible to showcase Suricata with this new release.

All components have been upgraded in this release to the latest version available but this is not the main improvement. SELKS is now able of doing Full Packet Capture thanks to Suricata and Moloch and benefit from an upgraded Scirius CE adding a new threat hunting interface.

Alert metadata in Scirius Hunting interface

Moloch addition allows the user to investigate and explore captured data via the Moloch viewer that provide an intuitive interface. The new Scirius threat hunting interface proposes a drill-down approach that allow to quickly find relevant alerts in a haystack and start investigation by what matter.

Features, fixes and major improvements:

  • The whole stack has been upgraded
    • Over 21 new dashboards
    • Hundreds of visualizations
    • New Threat Hunting interface
    • Full Packet Capture possibility
  • Elasticsearch 6.7.1
  • Logstash 6.7.1
  • Kibana 6.7.1
  • Moloch 1.8.0  –  The new SELKS makes use of Moloch and Moloch viewer to parse and view the full packet capture done by Suricata. Moloch comes with an arsenal of tools and features on its own like:
    • CyberChef
    • Extremely flexible and easy to use interface for FPC drill down, filtering, search and pcap export
  • Scirius 3.2.0 CE
      • Threat Hunting based on Suricata’s alerts metadata
      • Administration, ruleset and threat hunting management
      • Any field and action are selectable and searchable
      • Order and set up your own threat hunting dashboard in seconds with drag and drop functionality

TLS Server Name Identification

HTTP UserAgent selection

 

Easily select and filter on any metadata

Easily select and filter on any metadata

 

  • Suricata  – latest git edition anytime available.
  • SELKS scripts upgrade
    • available now system wide in “/usr/bin”
    • Full packet Capture retention policy – thanks Joren0494 !
    • selks-health-check_stamus  – SELKS health check script
  • Debian – always thankful !
  • EveBox – always the latest and very thankful for your support and extremely fast bug fixing and feature addition

More  screenshots of SELKS 5 release 

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Download

To download SELKS 5, pick one of the two flavors:

SELKS with desktop
  • HTTP: SELKS-5.0-desktop.iso
  • Sha256sum: 60c52286df9d1d250efac3f24644bd5b59bf5728d2c50bd722d8e4c9e8ce2089
SELKS without desktop

Usage

You can find the first time set up instructions on our SELKS 5.0 wiki page.

SELKS 4 user can upgrade their running systems using the following Upgrade instructions.

Feedback is welcome

Any feedback as always is greatly appreciated! 🙂

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested please make sure you try it in your test/QA set up first.

Thank you!

 

0

Hi!
Yet another upgrade of our SELKS. We are very thankful to all the great  Open Source projects and tools for making it possible to showcase Suricata with our new distro.
Features and fixes post SELKS 5 Beta :

  • Elasticsearch 6.5.3
  • Logstash 6.5.3
  • Kibana 6.5.3
  • Moloch 1.6.2  –  The new SELKS makes use of Moloch and Moloch viewer to parse and view the full packet capture done by Suricata. Moloch comes with an arsenal of tools and features on its own like:
    • CyberChef
    • Extremely flexible and easy to use interface for FPC drill down, filtering,search and pcap export
  • Scirius 3.1.0 CE
    • Administration, ruleset and threat hunting management
    • Blazing fast drill down and search capability through millions of events with milliseconds response time
    • Easy filter and grouping of alerts
    • Any field and action is selectable and searchable
    • Select or negate filter
    • Order and set up your own threat hunting dashboard in seconds with drag and drop functionality
    • Scirus Alerting rules event details

      Scirus Alerting rules event details

  • Suricata  – always latest git edition and features available.
  • SELKS scripts upgrade
    • available now system wide in “/usr/bin”
    • Full packet Capture retention policy – thanks Joren0494 !
  • Thank you for all the major community contributors form the community
  • Debian – always thankful !
  • EveBox – always the latest and very thankful for your support and extremely fast bug fixing and feature addition

Read more about the features and browse through screenshots of major SELKS 5 release 

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Download

To download SELKS 5, pick one of the two flavors:

SELKS with desktop
  • HTTP: SELKS-5.0RC1-desktop.iso
  • MD5sum: 192aa38436dcee6c98a6ae36d9e3b7df
  • Sha1sum: f48c0fe1edaaa8817c0a9043cb29e3edee4af93e
  • Sha256sum: 9f55a9ff4ee5c4c3c67646d0d5ae4e343f01f6abaf8e433ee9e3e78426c2f3e7
SELKS without desktop
  • HTTP: SELKS-5.0RC1-nodesktop.iso
  • MD5sum: 27733887bd1ad20c61d9be4973a66074
  • Sha1sum: dde637f8639254879ada06b9b68e691c3c904748
  • Sha256sum: b32370a35785f336d863d763372820ec13987c3a83a974f26d849eb81f721f4f

Usage

You can find the start instruction including the initial setup script usage on SELKS 5.0 wiki page.

SELKS 4 user can upgrade their running systems using the following Upgrade instructions.

Feedback is welcome

Any feedback as always is greatly appreciated! 🙂

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested please make sure you try it in your test/QA set up first.

Thank you!

 

0

Hey! Our new and upgraded showcase for Suricata has just been released – SELKS5 Beta. Thanks to lots of help from the community and dev work we are pleased to announce the first beta release of our new SELKS5.

Our major new features and additions include :

  • Suricata IDS/IPS/NSM 4.1-dev – latest Suricata packaged with new and enabled features like
    • Full Packet Capture enabled on SELKS  – yes, Suricata can do FPC as well.
    • Rust enabled
      • new protocols SMBv1/2/3, NFSv4, Kerberos, FTP, DHCP, IKEv2
      • more possibility for file extraction – SMTP/HTTP/SMB/NFS/FTP
    • Hyperscan enabled for extra performance boost.
  • Major upgrade from Elasticsearch/Kibana/Logtsash (ELK) 5.x to the ELK 6 stack making available a ton of new features and enhancements.
  • Scirius 3.0
    • New Hunt interface allowing for fast drill down approach enabling of filtering out the noise and concentrating on threats in seconds
    • Grouped rules factorization via usage of IP reputation feature of Suricata

  • Evebox – bugfixes and parsing improvements.
  • Debian – our favorite OS
  • Moloch  –  The new SELKS makes use of Moloch and Moloch viewer to parse and view the full packet capture done by Suricata. Moloch comes with an arsenal of tools and features on its own like:
    • CyberChef
    • Extremely flexible and easy to use interface for FPC drill down, filtering,search and pcap export

As always we are very thankful to the above Open Source projects and tools for making it possible to showcase Suricata and our new distro

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Download

To download SELKS 5, pick one of the two flavors:

SELKS with desktop
  • HTTP: SELKS-5.0beta1-desktop.iso
  • MD5sum: af4ae135dd60baea7183ac5bdb4a5863
  • Sha1sum: 878348effeefda387677002cb0d1aab529752ad3
  • Sha256sum: d6cf5e0bd583315e9b10229a1c73938087e3377997317ceed508fc55e5239c19
SELKS without desktop
  • HTTP: SELKS-5.0beta1-nodesktop.iso
  • MD5sum: 3bfbb8cf626f0f2979f02148c2bad4f5
  • Sha1sum: 80d0b855608ad458781478d1e2e9fd41c56b0c06
  • Sha256sum: 34019555e07e0cf47b3fb1e260f7c0b024553267338f02df8f949a1ef208741f

Usage

You can find the start instruction including the initial setup script usage on SELKS 5.0 wiki page.

SELKS 4 user can upgrade their running systems using the following Upgrade instructions.

Visual tour

Some visuals to give you a glimpse of the things you can do with SELKS.

Scirius landing page - Administer, Hunt, Search, Drill down and filter, Correlate events and FPC

Scirius landing page – Administer, Hunt, Search, Drill down and filter, Correlate events and FPC

21 ready to use Kibana dashboards consisting of over 200 visualizations

21 ready to use Kibana dashboards consisting of over 200 visualizations

Moloch Suricata Plugin

Moloch Suricata Plugin

Moloch and CyberChef navigation, drill down and display

Moloch and CyberChef navigation, drill down and display

TLS GeoIP and sni breakdown

TLS GeoIP and sni breakdown

TLS version and sni

TLS version and sni

TFTP GeoIp and events over time

TFTPGeoIp and events over time

SSH proto fields and geoIP visualizations

SSH proto fields and geoIP visualizations

SMTP Geoip events

SMTP Geoip events

SMB Proto fields

SMB Proto fields

SMB Alert trends

NFS protocol fields visualizations

NFS protocol fields visualizations

KRB5 protocol fields visualizations

KRB5 protocol fields visualizations

KRB5 alerts trending, sources and GeoIP

KRB5 alerts trending, sources and GeoIP

IKEv2 GeoIP and events trending

IKEv2 GeoIP and events trending

IKEv2 protocol fields break down

IKEv2 protocol fields break down

NSM and IDS time series

NSM and IDS time series

Rich HTTP details correlation and FPC

Rich HTTP details correlation and FPC

HTTP protocol data and GeoIP visualizations

HTTP protocol data and GeoIP visualizations

Fileinfo break don by protocols

Fileinfo break don by protocols

DNS protocol visualizations by fields

DNS protocol visualizations by fields

DNS Heat maps

DNS Heat maps

DNP3 event details correlation and FPC

DNP3 event details correlation and FPC

DNP3 protocol fields and sources info

DNP3 protocol fields and sources info

DHCP protocol fields visualizations, events correlation and FPC availability

DHCP protocol fields visualizations, events correlation and FPC availability

Application layer protocols breakdown

Application layer protocols breakdown

Application layer protocols breakdown -2

Application layer protocols breakdown -2

Application layer protocols breakdown -3

Application layer protocols breakdown -3

Per VLAN details and visualizations

Per VLAN details and visualizations

Per alert event details, metadata, correlation and FPC

Per alert event details, metadata, correlation and FPC

Helpful NSM birds eye views and selections

Helpful NSM birds eye views and selections

Alert event break down by protocol and GeoIP visualization

Alert event break down by protocol and GeoIP visualization

TrafficID

TrafficID

Moloch visualizations, easy filtering and drill down

Moloch visualizations, easy filtering and drill down

Moloch per flow/session visualizations, easy filtering and drill down

Moloch per flow/session visualizations, easy filtering and drill down

 

Feedback is welcome

Any feedback as always is greatly appreciated! 🙂

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested please make sure you try it in your test/QA set up first.

Thank you!

 

 

 

 

Following the release of Scirius Community Edition 2.0, Stamus Networks is happy to announce the availability of Scirius Enterprise Edition U29. It is using the ruleset management capabilities of Scirius CE 2.0 so new features such as transformations and public sources are available.

This release continues on the redesign of the interface done with Scirius CE. The landing page for the appliances management has been modified to offer a list of appliances with a number of filtering and ordering options.

This list has expandable items so it is easy to get information about one specific probe:

The asynchronous tasks display has also been redesigned with the same consistent approach:

If we have been busy in the design, the U29 release also comes with three exciting new functional features: REST API, VPN based probes and device monitoring dashboards.

The REST API is allowing third party application to query and modify the objects defined in scirius:

Applications like SIEM would benefit of that as it will enable powerful integration.

The VPN based probes is a big change as it allows to have probes that can connect to SEE from behind private networks/NAT/Firewalls. There is no need anymore of direct connectivity from Scirius to the probe.

The monitoring dashboard is available for Scirius Enterprise itself and for the managed Stamus Probes. It gives key indicator of the health of the devices:

Feel free to contact us if ever you want more information about our products. We will be happy to set up a demo and answer any of your questions.

Stamus Networks is proud to announce the availability of Scirius Community Edition 2.0. This is the first release of the 2.0 branch that features a brand new user interface and new features such as lateral movement and target transformations. Both modify signatures to improve them. Lateral movement uses an algorithm to enlarge the signature IP address filter to detect attacks in the internal networks. Target transformation implement an other algorithm to add target keyword to signatures thus helping to find and visualize attack paths.

Scirius 2.0.0 now features an automated addition of any of the sources defined in the public ruleset list published by the OISF:

So you can now add to your ruleset a new feed/source in two clicks. That’s really easier compared to the form based method where a series of fields as to be entered. The addition process itself is also faster. The parsing and update time of a ruleset like ET Pro has been improved to be three times faster in this version.

As you may have noticed, Scirius 2.0.0 interface is really different from one from the previous versions:

Scirius is now using the Patternfly framework to provide a consistent interface and usability oriented components. Usability has also been improved by the integration of the documentation in the interface.

On Suricata related side, the most important change is the handling of transformations. Scirius can now modify the signatures through a transformation:

Currently two transformations are available and they aim at making Suricata’s detection capabilities stronger:

Lateral Movement

Lateral movement transformation modifies signatures to have them detect lateral movement. As signatures are often written with the EXTERNAL_NET and HOME_NET variables, this means they won’t match if both sides of a flow are in the HOME_NET. Thus, lateral movements are not detected. This transformation changes EXTERNAL_NET to any to be able to detect lateral movements. Scirius propose per ruleset, per categories and per signature changes. One of the value proposed is auto that use an algorithm that trigger the substitution if the signature verifies some properties.

Target Keyword

The second substitution is the addition of the target keyword donated by Stamus Networks. Available since Suricata 4.0, the target keyword can be used to tell which side of a flow triggering a signature is the target. If this key is present then related events are enhanced to contain the source and target of the attack. Once more the user can choose the value of the option or let Scirius determine what side to use via an algorithm using signature properties.

For the eye candy fans, pktcity is now part of Scirius. This 3D webGL visualization interface is now available as part of the new dashboards:

Finally, for the list addicts, here is Scirius 2.0.0 changelog:

  • Rule transformation with lateral movement and target
  • Support of OISF public sources for easier setup
  • Convert documentation to sphinx and integrate it in interface
  • Rework of interface with Patternfly components
  • Link to Onyphe to get IP informations
  • Rules parsing optimization
  • More dashboards including pktcity webGL visualization
  • Initial REST API to interact with Scirius from outside

Scirius 2.0.0 is available on github. Debian packages for SELKS are also available. Users of Scirius Enterprise Edition will get access to this feature in the upcoming 29 release.

0

This first edition of SELKS 4 is available from Stamus Networks thanks to a great and helpful feedback from our open source community – Thank you! This new major release features a version jump for all the main software stacks. Suricata switches from 3.2 to 4.0, Elastic stack is ugpraded from 2.5 to 5.5 and even Debian is now Stretch, the latest stable release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major new release featuring all components upgrade and of course latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 4.0.x – latest Suricata packaged with Hyperscan enabled for extra performance boost. The latest edition of Suricata among many fixes and improvements includes:
    • extra alert data like for example http body added to the alert json logs wherever available
    • protocol renegociation which means STARTTLS and CONNECT support
  • Major upgrade from Elasticsearch/Kibana/Logtsash (ELK) 2.x to the ELK 5 stack making available a ton of new features and enhancements.
  • Scirius 1.2.4 – bugfixes, better correlation capability with EveBox and introduction of IPS rules support.
  • Evebox – many new features including reporting and comments on the log events.
  • Debian Stretch – All new OS features, kernel and tools.

As always – as a Stamus Networks extra sauce the latest stable kernel (4.12.8 at the time of this writing) is available for install if you wish.

Download

To download SELKS 4:

  • SELKS with desktop: Torrent, HTTP (MD5sum: 70783e4d441932103c3410c0b778b401)
  • SELKS without desktop: Torrent, HTTP (MD5sum: 335e31cd2b3a864f432c7d57efe007cd)

Usage

To remotely access the web management interface :

  • https://your.selks.IP.here/ – Scirius ruleset management and a central point for all dashboards and EveBox alert and event management.

Usage and logon credentials (OS and web management user)

  • user: selks-user
  • password: selks-user (password in Live mode is live)

The default root password is StamusNetworks

Visual tour

Some visuals to give you a glimpse of the things you can do with SELKS.

Scirius – ruleset manager and dashboard central management console.

Scirius – rule availability by ruleset information.

Scirius- “google” search your rules

Dashboards – mail attachments

Dashboards – mail application supplemental info

Dashboards – DNS geoip heat map

Dashboards – VLAN supplemental info

Dashboards – availability of full events correlation via EveBox and Scirius

Dashboards – extra http data for better visibility.

Dashboards – ssh data available for drill/break downs as well.

Dashboards – dns events at a glance

Dashboards – alert supplemental log information.

EveBox reporting

Dashboards – valuable break down of alert data information.

Dashboards – break down of http user agents that have generated alerts

EveBox – alert comments availability.

 

 

Howto

Upgrade from SELKS 3

To upgrade your existing SELKS 3 to SELKS 4 preview, please refer to SELKS-3.0-to-SELKS-4.0-upgrades wiki page.

Create your own ISO

SELKS 4 is available for download ready to use (as explained at the beginning of the article).

However – if you want to you can create and/or customize your own SELKS 4 ISO

Once installed
  • Please refer to Initial Setup section of the documentation
  • Keep your SELKS up to date
  • Recommended initial set up for SELKS 4.0 is 2CPUs 5-6Gb RAM
  • If you need to reset/reload all the dashboards  – you can do like so
    • In Scirius on the top left corner drop down menu select System Settings
    • click on the Kibana tab
    • choose Reset SN dashboards

Feedback is welcome

Any feedback as always is greatly appreciated! 🙂

Give us feedback and get help on:

Thank you!

0

Suricata 4.0 is out and this switch from 3.x to 4.x is not marketing driven because the changes are really important. This post is not exhaustive on changes. It is Stamus Networks’ take on some of the important changes that have been introduced in this version.

Rust addition

This is the big step forward on the technology side. Suricata is written in C language. This gives performances and a good control over memory. But it goes with a series of well known problems. I name here buffer overflows, use after free, …

And the worse is that Suricata is parsing traffic content which is a kind of vice supercharged user input. If one should not trust user input, guess how careful we should be with network traffic. At Suricon 2016, Pierre Chifflier did present a proof of concept implementation of protocol parsers in Rust. The idea is to use the property of Rust that has been designed to avoid complete class of attacks on memory handling. But there is more in the approach as the implementation is using Nom which is a Rust parser combinator framework. It allows you to write protocol parser easily and in a reusable way. Thus the addition of Rust is two things at the same time: more security and easier code. Which means a lot of new protocols should be added in the near future.

Suricata 4.0 Rust support comes with NFS, DNS and NTP. NTP support is implemented via an external crate (read library): ntp-parser.

As mentioned before, the code uses Nom and the syntax is very different from traditional code. For instance, here is the code of ntp-parser parsing NTP extension:

named!(pub parse_ntp_extension,
    do_parse!(
           ty: be_u16
        >> len: be_u16 // len includes the padding
        >> data: take!(len)
        >> (
            NtpExtension{
                field_type:ty,
                length:len,
                value:data,
            }
        ))
);

This define a parsing function that read the stream of data. The code says, take 16 bits, store them as unsigned integer in ty. Then store the next 16 bits as unsigned integer in len. Then store in data a chunck of data of length len. And with that build a NTP extension structure. If the writing is concise and efficient, the best thing with Nom is under the hood. Nom is taking care of detecting the invalidities. For instance we could have a chunck of data of length 50, and len being set to 1000 (remember Heartbleed ?). Nom will see that there is not enough data available in the chunck and return it wants more data.

Better alerts

As you may know, the preferred output of Suricata is the EVE JSON format. It is flexible, easy to extend and easy to read by human and tools. Suricata 4.0 is introducing some major changes here:

  • ‘vars’ extraction mechanism
  • The new target keyword
  • HTTP bodies logging
HTTP body output

Suricata is able to uncompressed HTTP body on the fly and match on the uncompressed content. This means that if you get the payload of the stream triggering the alert in your event, you will just see compression noise and won’t be able to analyze why the alert was triggered. Suricata is now able to include the HTTP bodies in the alert. The analyst can then directly see from the event the content that did trigger the alert.

The following event shows how payload_printable is completely compression noise and the http_response_body_printable is readable:

Target keyword

The new target keyword is a fix on a very old problem. It is not possible to know in an alert event which side of the source or destination is the target of the attack. This is a problem as it is not possible to automate things due to that lack of information. The target keyword allow the rules writer to specify which is side is the target. Doing so automated analysis and better visualization can be made.

Usage is simple, signature has to contain the target keyword with value dest_ip or src_ip. For example, in a simple scan alert we have:

alert tcp $EXTERNAL_NET any -> $HOME_NET 3306 (msg:"ET POLICY Suspicious inbound to mySQL port 3306"; flow:to_server; flags:S; threshold: type limit, count 5, seconds 60, track by_src; reference:url,doc.emergingthreats.net/2010937; classtype:bad-unknown; target: dest_ip; sid:2010937; rev:2;)

If target is present in a signature, the alert is added an alert.source and alert.target field:

For example, on a visualization where node are IP address and links are alerts between the two, we can get an idea of the possible compromised path. With the target addition, we can switch from a non oriented graph:

To an oriented graph that show which paths were really possible:

If you know French, you can learn more about this subject with Eric Leblond’s talk at SSTIC 2017.

Vars extraction

This is one of the most expected feature of Suricata 4.0. This has been described by Victor Julien in an extensive blog post. The concept is to be able to define in signature data to extract and store them in a key value form. There is a lot of possible usage ranging from application version extraction to getting exfiltred data. For example, let’s consider there is a domain we are interested in. One interesting information is the list of email addresses where mail are sent to. To do so we can use the following signature:

alert smtp any any -> any any (msg:"Mail to stamus"; content:"rcpt to|3A|"; nocase; content:"stamus-networks.com"; within: 200; fast_pattern; pcre:"/^RCPT TO\x3a\s*<([\w-\.]+@stamus-networks.com)>/ism pkt:email"; flow:established,to_server; sid:1; rev:1;)

The magix here is the groupe in the regular expression ([\w-\.]+@stamus-networks.com) that is save in a packet var named email by the pkt:email in the regular expression definition.

Using that signature we get this kind of alerts:

The key point here is the vars sub object:

  "vars": {
    "pktvars": [
      {
        "email": "eleblond@stamus-networks.com"
      }
    ]
  },

We have an extraction of the data and this can be easily search by tool like Elasticsearch or Splunk.

Conclusion

Suricata 4.0 is really an important milestone for the project. Introduction of Rust is opening a really interesting path. The alerts improvement may change the way signatures are written and it will help to provide really accurate information to the analysts.

Suricata 4.0 is already available in SELKS and it will be available in Stamus Probe by the end of August. To conclude on a personal note, we, Stamus Networks, are really happy to have contributed to this release with features such as via HTTP body logging and target keyword.

0

After a very valuable round of testing and feedback from the community  we are pleased to announce the SELKS 4 RC1 availability.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 4.0.x – latest git master Suricata packaged with Hyperscan enabled for extra performance boost. This edition of Suricata besides many improvements and bug fixes also includes extra alert data like for example http body added to the alert json logs wherever available.
  • Elasticsearch 5.5.0  – part of the ELK5 stack upgrade making available a ton of new features and enhancements.
  • Logstash 5.5.0 – performance improvement over 2.x and ES5 compatibility.
  • Kibana 5.5.0 – taking advantage of the latest dashboarding features of ES.
  • Scirius 1.2.2 – bugfixes, better correlation capability with EveBox and introduction of IPS rules support.
  • Evebox – many new features including reporting and comments on the log events.
  • Debian Stretch – All new features, kernel and tools.

EveBox

Alert event with a comment field.

Kibana

Verbose HTTP logging

Kibana

GeoIP heat maps

EveBox

Supplemental alert data logging

 

Download

To download SELKS4-RC1:

Usage

Usage and logon credentials (OS and web management user)

  • user: selks-user
  • password: selks-user (password in Live mode is live)

The default root password is StamusNetworks

To remotely access the web management interface :

  • https://your.selks.IP.here/ – Scirius ruleset management and a central point for all dashboards and EveBox alert and event management.

Howto

Upgrade

To upgrade your existing SELKS 3 to SELKS 4 preview, please refer to SELKS-3.0-to-SELKS-4.0-upgrades wiki page.

It is recommended to follow the onscreen instructions and if needed answer “yes” to all changes. At the end of the upgrade you will be asked to enter the interface that you will use for IDS/sniffing. Please enter (eth0 for example) the interface name and reboot when the script is done.

Create your own ISO

To create your own SELKS 4 preview ISO (if your host OS is Jessie):

git clone https://github.com/StamusNetworks/SELKS.git
git checkout SELKS4-dev
./install-deps.sh
cd /usr/share/live/build/data/debian-cd/ && ln -s squeeze stretch
./build-debian-live.sh

It will take probably 30-40 min and you should end up with the SELKS.iso under the Stamus-Live-Build folder.

Once installed/upgraded
  • Please feel free to choose the IDS sniffing/listening interface either via the desktop icon Setup-IDS-Interface or via the cmd calling /opt/selks/Scripts/Setup/setup-selks-ids-interface.sh
  • Any further upgrades are done via a wrapper script located in /opt/selks/Scripts/Setup/selks-upgrade_stamus.sh
  • Recommended set up for SELKS 4.0RC1 is 2CPUs 5-6Gb RAM
  • If you need to reset/reload all the dashboards  – you can do like so
    • In Scirius on the top left corner drop down menu select System Settings
    • click on the Kibana tab
    • choose Reset SN dashboards

Feedback is welcome

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested and aims at upgrading your current SELKS 3.0 to  SELKS 4.0RC1 please make sure you try it in your test/QA set up first and give us any feedback.

Thank you!

Stamus Networks is proud to announce the availability of Scirius 1.2.0. This release of our Suricata ruleset management interface comes after 4 months of development bringing two new major features: rules transformations to manage IPS and users activity logging to ease collaboration.

Rules transformation

With rules transformations, Scirius can now manage Suricata in IPS mode but also add the filestore option to specific rules allowing the user to transform existing rules coming from feed in rules realizing file extraction.

A signature can be transformed per ruleset to a drop or reject rule as shown in the following capture:

The filestore transformation will trigger file extraction by Suricata in case of alert. This allows user to have file extraction without the need of cloning existing rules.

User activity logging

The second big new feature is user activity logging. It is now possible to comment actions. A team collaboring on the same Scirius can now comment actions such as disabling a rule or adding a threshold.

It is also possible to simply comment on a rule.

All these features are already available in Scirius Enterprise and Amsterdam and will be available in SELKS in the coming days.

Eric Leblond gave a talk entitled “The adventures of a Suricata in eBPF land” at netdev 1.2, the Technical Conference on Linux Networking. This talk reviewed Stamus Networks’ work in the field of bypass and showed how the eBPF technology can be used to implement this feature.

eBPF is a technology that extends the traditional Berkeley Packet Filter that you can for example use with tcpdump. For instance eBPF filter can be written in a subset of C and allows kernel and userspace to share data via maps that can be for example an array or hash table. This technology has been used to implement a kernel bypass in Suricata. The idea is that Suricata is asking the Linux kernel to stop sending  it (bypass) packets for particular flow once it has decided that no further inspection is needed to be done.

For detailed information on the subject, you can get the Slides of “Suricata and eBPF” or watch the video that is already available thanks to the great work of Netdev team:

0

Introduction

Stamus Networks was working on a new Suricata feature named bypass. It has just been merged into Suricata sources and will be part of the upcoming 3.2 release. Stamus team did initially present his work on Suricata bypass code at Netdev 1.1, the technical conference on Linux networking that took place in Sevilla in February 2016.

In most cases an attack is done at start of TCP session and generation of requests prior to attack is not common. Furthermore multiple requests are often not even possible on same TCP session. Suricata reassembles TCP sessions till a configurable size (stream.reassembly.depth in bytes). Once the limit is reached the stream is not analyzed.

Considering that Suricata is not really inspecting anymore the traffic, it could be interesting to stop receiving the packets of a flow which enter in that state. This is the main idea behind bypass.

The second one consist in doing the same with encrypted flows. Once Suricata sees a traffic is encrypted it stops inspecting it so it is possible to bypass the packets for these flows in the same way it is done for packets after stream depth.

In some cases, network traffic is mostly due to session we don’t really care about on the security side. This is for example the case of Netflix or Youtube traffic. This is why we have added the bypass keywords to Suricata rules language. A user can now write a signature using this keyword and all packets for the matching flow will be bypassed. For instance to bypass all traffic to Stamus Networks website, one can use:

alert http any any -> any any (msg="Stamus is good"; content:"www.stamus-networks.com"; http_host; bypass; sid:1; rev:1;)

This is for sure just an example and as you may have seen our website is served only on HTTPS protocol.

Currently, Netfilter IPS mode is the only capture supporting the bypass. Stamus team represented by Eric Leblond will be at Netdev 1.2, first week of October 2016, to present an implementation of bypass for the Linux AF_PACKET capture method based on extended Berkeley Packet Filter.

And if you can’t make it to Japan, you will have another chance to hear about that during suricon, the Suricata user conference that will take place in Washington DC beginning of November.

Suricata bypass concepts

Suricata bypass technics

Suricata is now implementing two bypass methods:

  • A suricata only bypass called local bypass
  • A capture handled bypass called capture bypass

The idea is simply to stop treating packets of a flow that we don’t want to inspect anymore as fast as possible. Local bypass is doing it internally and capture bypass is using the capture method to do so.

Test with iperf on localhost with a MTU of 1500:

  • standard IPS mode: 669Mbps
  • IPS with local bypass: 899Mbps
  • IPS with NFQ bypass: 39 Gbps
Local bypass

The concept of local bypass is simple: Suricata reads a packet, decodes it, checks it in the flow table. If the corresponding flow is local bypassed then it simply skips all streaming, detection and output and the packet goes directly out in IDS mode and to verdict in IPS mode.

Once a flow has been local bypassed it is applied a specific timeout strategy. Idea is that we can’t handle cleanly the end of the flow as we are not doing the streaming reassembly anymore. So Suricata can just timeout the flow when seeing no packets. As the flow is supposed to be really alive we can set a timeout which is shorter than the established timeout. That’s why the default value is equal to the emergency established timeout value.

Capture bypass

In capture bypass, when Suricata decides to bypass it calls a function provided by the capture method to declare the bypass in the capture. For NFQ this is a simple mark that will be used by the ruleset. For AF_PACKET this will be a call to add an element in an eBPF hash table stored in kernel.

If the call to capture bypass is successful, then we set a short timeout on the flow to let time of already queued packets to get out of suricata without creating a new entry and once timeout is reached we remove the flow from the table and log the entry.

If the call to capture bypass is not successful then we switch to local bypass.

The difference between local and capture bypass

When Suricata is used with capture methods that do not offer the bypass functionality of eBPF/NFQ mark – pcap, netmap, pfring – it will switch to local bypass mode as explained above. Bypass is available for Suricata’s IDS/IPS and NSM modes alike.

Handling capture bypass failure

Due to misconfiguration or to other unknown problems it is possible that a capture bypassed flow is sending us packets. In that case, suricata is switching back the flow to local bypass so we handle it more correctly.

0

Yes, we did it: the most awaited SELKS 3.0 is out. This is the first stable release of this new branch that brings you the latest Suricata and Elastic stack technology.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Suricata page in Scirius

Suricata page in Scirius

Main changes and new features

Suricata 3.1.1

SELKS 3.0 comes with latest Suricata namely 3.1.1 bringing a big performance boost as well as some new IDS and NSM capabilities.

Elasticsearch 2.x and Kibana 4

But the main change in SELKS 3.0 is the switch to the latest generation of the Elastic stack. On user side this means Kibana 3 has been replaced by Kibana 4. And this really means a lot. Kibana 4 is a complete rewrite of Kibana 3 being non backward compatible on data side. So, our team had to redo from scratch all dashboards and visualizations. The result is a new set of 11 ready-to-use dashboards and a lots of visualizations that you can use to build your own dashboards.

Kibana Alert dashboard

Kibana Alert dashboard

correlate-alerts

Complete flow and rule correlation view of an alert

Latest Scirius Community Edition

On the ruleset management side, SELKS 3.0 comes with Scirius Community Edition 1.1.10 that has support for advanced Suricata feature like xbits.

Thresholding

Suppression with Scirius

Thresholding-1

Threshold and suppress ruleset view with Scirius

Thresholding-2

Thresholding with Scirius

Scirius CE also brings thresholding and suppression support as well as an integrated backup system which allows for back up to be done (besides locally) in locations such as :

  • FTP
  • Amazon AWS
  • Dropbox
Evebox

SELKS 3.0 comes with Evebox an alert management/viewer/report interface for Suricata that presents events as a mailbox to provide classification via acknowledgement and escalade.

Mailbox view in Evebox

Mailbox view in Evebox

One of the other interesting features of Evebox is the capability to create and export pcap generated from events:

Pcap-1

Payload pcap generation (Evebox)

Pcap-2

Payload pcap generation (Evebox)

Features list

  • Suricata IDS/IPS/NSM  – Suricata 3.1.1 packaged.
  • Elasticsearch 2.3.5  – latest available ES edition featuring speed, scalability, security improvements and more.
  • Logstash 2.3.4 – performance improvement ES 2.3 compatability, dynamically reload pipelines on the fly and more
  • Kibana 4.5.4 – taking advantage of the latest features and performance improvement of ES
  • Scirius 1.1.10 – support for xbits, hostbits, thresholding, suppression, backup and more
  • Evebox – alert management/viewer/report interface for Suricata/ES  allowing easy export of payload/packets into pcaps
  • 4.4.x longterm kernel – SELKS 3.0 comes by default with 4.4.16 kernel.
  • Dashboards – reworked dashboards with flow and rule correlation capability.

SELKS comes with 11 ready to use Kibana dashboards. More than 190 visualizations are available to mix, match, customize and make your own dashboards as well.

Please feel free to try it out, spread the word, feedback and let’s talk about SELKS 3.0.

To get you started

Once downloaded and installed, you can get access to all components via https://your.selks.IP.here/

The default user and password for both web interface and system is:

  • user: selks-user
  • password: selks-user

The default root password is StamusNetworks.

Please note that in Live mode the password for the selks-user system user is live.

Upgrades

There is no direct upgrade path from SELKS 2.0 to SELKS 3.0 due to a number of breaking and compatibility changes in Elasticsearch 1.x to 2.x and Kibana 3.x to 4.x. The only proposed upgrade path is SELKS 3.0RC1 upgrade to SELKS 3.0

More about SELKS 3.0

Stamus Networks is proud to announce the availability of version 1.0, nicknamed “glace à la vanille”, of Amsterdam, our container based ready to use Suricata IDS. Amsterdam is a fully web managed software appliance that is using Docker to provide:

  • Network Intrusion Detection and Network Security Monitoring via Suricata
  • Log storage and analysis via the Elastic stack: latest Logstash, Elasticsearch and Kibana are part of the Amsterdam
  • Suricata ruleset management and basic reporting via Scirius our web interface
  • Alerts listing and acknowledgement via Evebox

Scirius homepage

Each component is running in its own container and Amsterdam is using by default the official image on Docker Hub. This guarantees you fast update and heavily tested software. The orchestration of the different containers is done via Docker compose but all the details are hidden to you and Amsterdam should be your only interface in daily usage.

Installation is just a few commands:

pip install amsterdam
amsterdam -d ams -i wlan0 setup
amsterdam -d ams start

Once every containers are running, you can simply point your browser to https://localhost/ to start analyzing the traffic and fine tune the system. Kibana is coming with a set of predefined dashboards so you don’t have to build your own before starting to work.

Kibana Alert dashboard

Amsterdam offers you really easy upgrade via integrated commands:

amsterdam -d ams upgrade
amsterdam -d ams restart

Amsterdam is multi instances. For example, let’s say you have two customers where you analyzed the traffic when on site. You can set up two instances:

amsterdam -d customer1 -i wlan0 setup
amsterdam -d customer2 -i eth0 setup

and start the first one when at customer 1

amsterdam -d customer1 start

and second one when at customer 2

amsterdam -d customer2 start

The two different instances are not sharing any data, so you can freely show the interface to any of the customer if running the good instance. All data and configuration files are in customer1 directory for first customer and customer2 for the second one.

Amsterdam can digest any JSON formatted data. For that is is enough to copy a file to analyzed in the suricata directory inside the instance:

cp /path/to/passwords.json customer1/suricata/

This method makes it really easy to combine different sources of information into Kibana dashboards:
Pshitt and Suricata information

Amsterdam is also really easy to tune. The configuration files are stored for each components in the config directory so you can easily update Suricata, Logstash or Nginx configuration.

Stamus Networks is really excited by this first stable release of Amsterdam and we think that it has never been so easy to sniff and understand your network.

This release is dedicated to the memory of Edith Leblond.

0

After some hard team work, Stamus Networks is proud to announce the availability of SELKS 3.0RC1.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 3.0.x – latest git master suricata packaged.
  • Elasticsearch 2.3  – latest available ES edition featuring speed, scalability, security improvements and more.
  • Logstash 2.3 – performance improvement ES 2.3 compatability, dynamically reload pipelines on the fly and more
  • Kibana 4.5 – taking advantage of the latest features and performance improvement of ES
  • Scirius 1.1.6 – support for xbits, hostbits, thresholding, suppression, backup and more
  • Evebox – alert management/viewer interface for Suricata/ES  allowing easy export of payload into pcaps

SELKS comes with 11 ready to use Kibana dashboards using more than 190 visualisations.

Please feel free to try it out, spread the word, feedback and let’s talk about SELKS 3.0.

Thresholding-2

Thresholding with Scirius

Thresholding

Suppression with Scirius

Thresholding-1

Threshold and suppress ruleset view with Scirius

 

Pcap-1

Payload pcap generation (Evebox)

Pcap-2

Payload pcap generation (Evebox)

 

Dashboard-3

Dashboards

Dashboard-1

Dashboards

 

 

 

To get you started (the download link is below this paragraph):

Once installed in order to upgrade all components follow the guide here.

Usage and logon credentials (OS user)  – user: selks-user, password: selks-user (password in Live mode is live). The default root password is – StamusNetworks

Upon log in double click the Scirius icon on the desktop. Credentials are  – user: selks-user, password: selks-user. In the left upper corner click the drop down menu and choose “ALL” dashboards. Choose default index(click on logstash-* and then the green star) as depicted below. Then choose “Dashboards” and choose your desired dashboards from the 11 available.

enable-index-kibana

 

More about SELKS 3.0RC1

Stamus Networks is proud to announce the availability of Scirius 1.1.6. This new release brings interesting new features and a lot of bugfixes to our Suricata ruleset manager.

Rule page in scirius 1.1.6

The main new features in release are:

  • Backup support
  • Threshold support
  • Xbits and hostbits support
  • Down detection of scirius
  • Top src and destination in rule page
  • Fix of test system that takes Suricata local config into account

The backup system adds a set of new commands to manage.py to backup and restore completely a Scirius instance. scbackup will do a backup and screstore will erase everything and restore latest backup. Backup can be done locally but it is also possible to use FTP, Dropbox or Amazon AWS to store and fetch backups.

On the usability feature side the most important is the support of thresholding. Scirius is now managing a threshold.config that is used by Suricata to limit or suppress alert(s) for a signature under certain conditions. Easiest way to access this feature is to start from a rule page and look at new top source and destination tables:

Top src and dest IP for a signature

The arrow down and the cross can be clicked to trigger edition of a form for a threshold (limit) or a suppression. For instance if you click on the cross, you will get something like:
Suppression
If there is already a suppression activated for the network/IP, you get a warning:
Adding a suppression

Latest ruleset management feature is the handling of the new xbits and hostbits. When a rule is disable, all the rule sharing a flowbits, a xbits or a hostbits are also deactivated.

At last, browser is now detecting that Scirius is down allowing you to avoid to navigate away from a form you were editing till connection is not restored:
Scirius down

Scirius 1.1.6 may be a minor release for the number in term of features it adds a lots of things users were asking for. You can already get scirius 1.1.6 in latest amsterdam. And it will be part of SELKS 3.0 that will be available really soon.

0

Suricata stats in EVE JSON format

Suricata 3.0 will come with a lot of improvements on the output side. One of them is the ability to output Suricata internal statistics in the EVE JSON format.

Stats event in EVE JSON format

This functionality is already used by scirius to display statistics graphs of the Suricata running in SELKS, Amsterdam or Stamus Networks’ appliances:

Stats in scirius

This statistic sometimes help to visualize the impact of configuration change. For example, in the next screenshot the generic receive offloading on the capture interface has been disable at 23:33:

Impact of iface offloading

Impact is cristal clear as the counter of invalid decoding did stop increasing.

Using Kibana Timelion plugin

Amsterdam came with Kibana 4 and the Timelion plugin is preinstalled. Timelion is a plugin providing a new interface and language to graph timeline.

As Suricata stats data are fed into Elasticsearch, we can use it to graph Suricata performance data.

For example to graph DNS and HTTP memory usage, one can use the following syntax:

.es(metric=’avg:stats.dns.memuse’).label(‘DNS’) .es(metric=’avg:stats.http.memuse’).label(‘HTTP’)

Result is the following graph:
Screenshot from 2016-01-07 11-01-48

If you have a counter and want to graph rate, then you can use:

.es(metric=’avg:stats.capture.kernel_packets’).derivative().label(‘PPS’) .es(metric=’avg:stats.capture.kernel_drops’).derivative().label(‘Drops’)

And you get the following graph:

Screenshot from 2016-01-07 10-59-01

One interesting thing with Timelion is that you can use Lucene query to get a count of something really easily. For example to get a view on the rate of different event type, one can use:

.es(q=’event_type:http’) .es(q=’event_type:tls’) .es(q=’event_type:dns’)

Rate of different event types

Both method can be mixed so, if you have different probes (let’s say probe-1 and probe-2) you can do something like:

.es(q=’host.raw:”probe-1″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-1 DNS’) .es(q=’host.raw:”probe-2″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-2 DNS’)

Conclusion

The new Suricata statistic output is really improving the information we can use when doing performance analysis. Combined with timelion, we get a really easy and powerful solution. If you want to give a try to all these technologies one of the easiest way is to use Amsterdam which comes with latest Suricata and a pre installed timelion.

Stamus Networks is proud to announce the availability of the first technology preview of Amsterdam.

Amsterdam is a ready-to-use Suricata IDS/NSM based system running on Docker. It features the same components as SELKS our live and installable Suricata distibution. So by running Amsterdam you get:

  • Suricata: latest version of the IDS/NSM engine
  • Elasticsearch: the powerful search engine
  • Logstash: the data pipeline application injecting Suricata events in the database
  • Kibana: Version 4 of the famous dashboard interface
  • Scirius: Stamus Networks’ Suricata ruleset management interface

Each component runs in a separate container and Amsterdam is using the official images for ELK stack to provide you an always up-to-date experience. Amsterdam is available under GPLv3 license.

Using Amsterdam is really simple. Once installed, you need to setup a directory that will contain data and configuration. For example to create and use a directory named ams-wlan0 and sniff the wlan0 interface on host, one can run once:

amsterdam -d ams-wlan0 -i wlan0 setup

You can then start Amsterdam by running:

amsterdam -d ams-wlan0 start

Scirius running in Amsterdam

Once all containers are built and started, you can point your browser to http://localhost:8000 to get access to the management interface.

It is possible to run multiple instances of Amsterdam on the same system by using different data directories. Each data directory contains the configuration files of the components so you can easily tune your installation.

More information and source code available on Github.

0

 

Stamus Networks is proud to announce the availability of SELKS 2.1  release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major SELKS upgrade.

New Features

  • Elasticsearch 1.7  – upgrade from 1.5 (security fixes and faster recovery after restart)
  • Scirius 1.1 – upgrade from 1.0 (suricata and logstash performance stats)
  • Logstash 1.5.4 – upgrade from 1.4 (performance improvement
    in JSON handling and better security)

 

Some screenshot examples

Source addition page

Suricata memory usageElasticsearch and Logstash information

 

UPGRADE from SELKS 2.0

For those that use SELKS 2.0 and would like to do an in place upgrade to SELKS 2.1 you can follow THIS GUIDE.

NOTE: Please make sure that you test the upgrade in your test/QA environment first before doing it on your production systems.

Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

More about SELKS 2.1

Stamus Networks team is proud to announce the availability of Scirius 1.1. This new release brings a ton of new features:

  • Easier enabling/disabling of rules and categories
  • Delete events from Elasticsearch corresponding to all alerts of a given rule
  • Compatibility with Kibana 4
  • Add a graphic presenting Logstash insertion speed
  • Graphics with Suricata performance indicators
  • Greatly improved source addition
  • Preliminary support for Sourcefire ruleset
  • Validity checking of ruleset, imported sources and rules

One of the main new feature is the validity checking system. Scirius is now able to test whether a rule, a source or a ruleset will be valid for Suricata. This is not a simulated test as Scirius is using the configuration testing capability of Suricata to get the information.

One of the updated page is the the rule details page. It does now contain the validity of this rule respectively to the existing rulesets:
Validity check

In our example, the rule is invalid for first ruleset because the source containing the lua script is not active in the ruleset. In the second ruleset, the rules is not valid because the Suricata on the system is not supporting the not-yet-official(as of the moment of this writing) TLS extension for lua scripting. As you can see, the validity checking system is much more than a simple parsing and gives you a real view on the validity of rules on your own system. This system will work with any Suricata but it will give accurate information if the system is hosting the latest Suricata. This is due to the fact that Scirius is parsing the JSON console output of Suricata which has been added recently to the Suricata git master. Our team has packaged the latest Suricata and made it available in Stamus Networks repository to the users to give them the best of this new feature.

This testing system has been used to revamp the Source addition. Adding a source to an existing ruleset was complicated and involving multiple steps. Now, user can select to which rulesets a new source should be added when creating the source. So there is now basically one step:
Source addition form
The validity checking system will warn the user about a possible problem during the source creation:
Source addition page
As some rulesets may contain invalid signatures, user has the capability to choose whether or not he/she wants to ignore the detected problems.

The second main change in Scirius is its ability to graph some performance indicators. It is using the logstash metrics capability to get statistics on the insertion speed. This is a good indicator of the load of your logging system.

Elasticsearch and Logstash information

The second set of graphics is building performance indicators for Suricata. The Suricata page of Scirius is now able to graph:

  • Capture stats
  • Memory usage
  • Problem indicators

If capture stats is showing the usual drop and accept, the two others tabs are more interesting as they are showing the memory consumption of Suricata subsystem succh as TCP, Flow, DNS and HTTP protocols:
Suricata memory usage
The third tab is showing some Suricata indicators that can help to diagnose problem. All these indicators are extracted from Suricata statistics output in JSON format that will be part of Suricata 2.1.

Happy SELKS users can simply run apt-get update && apt-get dist-upgrade to get new version. Others can get it from Github.

0

Introduction

This is a short tutorial of how you can find and store to disk a self signed TLS certificates with Suricata IDPS and Lua (luajit scripting).

What does self signed TLS certificate mean – the quick version  from Wikipedia here. In other words – “certificate that is signed by the same entity whose identity it certifies” or anyone can create and deploy such a certificate. This kind of events are signed of some poorly setup TLS servers and that’s why it is good to keep an eye on such events in your network for the purpose of contingency monitoring – the very least.

TLS support in Suricata allows you to do match on TLS subject, TLS issuer DN and things like TLS fingerprint. For instance, one can do

alert tls any any -> any any (msg:"forged ssl google user";
            tls.subject:"CN=*.googleusercontent.com";
            tls.issuerdn:!"CN=Google-Internet-Authority"; sid:8; rev:1;)

to detect that the TLS issuer DN is not the one we were waiting for a given TLS subject. But there is no way to compare the two different fields. So how do we catch all those self signed certificates without knowing any details about them and/or any network/domain/port specifics either. And we want them all caught and stored to disk!

This case is one example where the Lua support in Suricata IDPS  shines – more than shines actually because it empowers you to do much more than chasing packet bytes with rule keywords and using PCREs inside a rule – and all those still deliver limited functionality in this particular scenario.

Lua and Suricata

Since version 2.0, Suricata has support for Lua scripting. The idea is to be able to decide if an alert is matching based on the return of a lua script. This script is taking some fields extracted by Suricata (the magic of it all) as parameters and return 1 in case of match and 0 if not. This lua scripting allows rules to implement complex logic that would be impossible with the standard rule language. For instance, Victor Julien was able to write a performant Heartbleed detection with lua scripting in the afternoon (the very same day) the problem/exploit was announced.

The syntax is the following, you prefilter with standard signature language, and you add a lua keyword with parameter being the script to run in case of partial match:

alert tls any any -> any any ( \
    msg:"TLS HEARTBLEED malformed heartbeat record"; \
    content:"|18 03|"; depth:2; lua:tls-heartbleed.lua; \
    classtype:misc-attack; sid:3000001; rev:1;)

For more information on Suricata Lua scripting, please read how to write luascripts for Suricata.

For self signed certificate detection, you need to write a script – shall we say “self-signed-cert.lua” and save it in your /etc/suricata/rules directory. Then you can use it in a rule like so –

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Now let us explain that in a bit more detail below.

At the time of this writing we are using this branch in particular – TLS Lua rebased . This is going to be merged to the Suricata git (latest dev) soon and later into beta, stable editions.

You need to make sure Suricata is compiled with Lua enabled:

# suricata --build-info
This is Suricata version 2.1dev (rev b5e1df2)
...
Prelude support:                         no
PCRE jit:                                yes
LUA support:                             yes
libluajit:                               yes
libgeoip:                                yes
...

In suricata.yaml make sure the tls section is enabled:

# a line based log of TLS handshake parameters (no alerts)
- tls-store:
  enabled: yes  # Active TLS certificate store.
  certs-log-dir: certs # directory to store the certificates files

We have the following rule file (self-sign.rules) content located in /etc/suricata/rules/ :

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Make sure you add the rule to your rules files loaded in suricata.yaml and that you copy the associated lua script in the same directory. Here you can find the self-signed-cert.lua script.

Then you can start Suricata IDPS the usual way you always do.

The active part of the lua script is the following:

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    
    if subject == issuer then
        return 1
    else
        return 0
    end
end

When suricata will see a TLS handshake (regardless of IP/port), it will run the Lua script. This one uses the fact that an equality between subject and issuer DN constitutes most of the self-signed certificates. When it finds such an equality it will return 1 and an alert will be generated.

This script is showing a really basic but useful code. However you can use all the power of Lua with the given info to do whatever you want/need.

The result

self-cert-extract

Extracted self signed SSL certificate

Some meta data about the certificate (in /var/log/suricata/certs/ you will find the .meta and pem  files):

TIME:              07/14/2015-14:45:16.757001
SRC IP:            10.0.2.15
DST IP:            192.168.1.180
PROTO:             6
SRC PORT:          49966
DST PORT:          443
TLS SUBJECT:       C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS ISSUERDN:      C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS FINGERPRINT:   80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18

 

Further more from the alert info in JSON format (/var/log/suricata/eve.json log) we also have extra TLS info for the generated alert :

{"timestamp":"2015-07-14T14:45:18.076794+0200","flow_id":137451536,"in_iface":"eth0",
"event_type":"alert","src_ip":"192.168.1.180","src_port":443,"dest_ip":"10.0.2.15","dest_port":49966,"proto":"TCP","alert":
{"action":"allowed","gid":1,"signature_id":999666111,"rev":1,"signature":"SURICATA TLS Self Signed Certificate","category":"Generic Protocol Command 
Decode","severity":3},"tls":{"subject":"C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS","issuerdn":"C=FR, ST=IDF, L=Paris, O=Stamus, 
CN=SELKS","fingerprint":"80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18","version":"TLS 1.2"}}

To get the extra TLS info in the JSON alert you need to enable it in the suricata.yaml like so:

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream
      filename: eve.json
      types:
        - alert:
            #payload: yes           # enable dumping payload in Base64
            #payload-printable: yes # enable dumping payload in printable (lossy) format
            #packet: yes            # enable dumping of packet (without stream segments)
            http: yes              # enable dumping of http fields
            tls: yes               # enable dumping of tls fields <<---
            ssh: yes               # enable dumping of ssh fields

so you can get all the JSON data in Kibana/Elasticsearch as well.

Conclusion

If you would like to learn more about what can you do with Lua and Suricata IDPS, those links below will put you off to a good start:

Suricata EVE JSON format is becoming the de-facto standard for this IDS. All type of events are now exported to this format. The JSON format allows a nice handling of data in external tool like Elasticsearch or even DOM. The output is readable by human but as an event/record can contain a lot of data it can be difficult to do a by-eye analysis when looking at a file. The following screenshot give you an idea of the possible output:

Tailing EVE

Using standard unix tools like grep on the EVE JSON file is not the perfect idea. For example if you want to extract a field to get some statistics you may want to try using grep, cut or awk but you may find it painful. And it is worthed to mention here that JSON fields are not ordered.

Here to the rescue comes the jq utility. jq is a tool dedicated to the transformation/parsing of a JSON entry. It is Debian packaged, so a simple apt-get install jq is enough for the install.

Some jq examples

The most basic usage is to colorize the entry. To do that, just do something like

$ tail -n100 eve.json| jq '.'

The output is done the pretty way:
JQ displaying an event
To get a one line per event output, just add the -c flag to the command:
One line

To extract a single field from the JSON events, one can do:

$ jq '.src_ip' eve.json
"58.218.211.155"
"58.218.211.155"
"58.218.211.155"

The point to remember is that the point in .src_ip is a place holder for the current entry.

By default when a field is not present null is displayed in the output. To fix that, it is possible to filter the event to only get the one we are interested in. This is done via the select keyword. For instance to select the SSH events and extract the information about the client part one can do:

$ tail eve.json | jq -c 'select(.event_type == "ssh")|.ssh.client'
{"proto_version":"2.0","software_version":"PUTTY"}
{"proto_version":"2.0","software_version":"PUTTY"}

Far more things can be done with jq. Good starting points are the jq manual and wiki.

0

Stamus Networks is proud to announce the availability of SELKS 2.0  release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major SELKS upgrade.

New Features

  • Debian Jessie based  – With Debian Jessie  being released last week  – 25 April   SELKS makes the switch as well (from Debian Wheezy and 3.2 kernel). The new Debian release Jessie will enable SELKS for much better HW compatibility, new kernel 3.16 and all the performance improvements, features and benefits with it right out of the box.
  • Elasticsearch 1.5  – upgrade from 1.4
  • Scirius 1.0 – upgrade from 1.0rc3

 

Some screenshot examples

Scirius

SELKS2.0-Scirius-2

Screenshot from 2015-04-20 22:07:08

IDS/IPS dashboards
SELKS2.0-1

12 ready to use IDS/IPS dashboards

VLAN3

By VLAN break down File Transactions/SSH

VLAN2

By VLAN break down DNS/TLS

SMTP-Attachments

By file attachment breakdown SMTP

VLAN1

By VLAN break down Alerts/HTTP

UPGRADE from SELKS1.2

For those that use SELKS 1.2 and would like to do an in place upgrade to SELKS 2.0 you can follow THIS GUIDE.

NOTE: Please make sure that you test the upgrade in your test/QA environment first before doing it on your production systems.

Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

More about SELKS 2.0

Stamus Networks is proud to announce the availability of Scirius 1.0. This is the first stable release of our web interface for Suricata ruleset management. It is providing an efficient way to manage and update the ruleset.

Scirius is displaying some graphics that will help you to get an idea of the activity of your Suricata probe and easily select rules that may be noisy and need to be deactivated:
Screenshot from 2015-04-20 22:07:08

But the main focus is on ruleset handling. You can for example follow the change of a signature source:

Screenshot from 2015-04-20 22:05:36

Scirius is not meant to replace a good dashboard interface, so it is providing a link to Kibana dashboard:
Screenshot from 2015-04-20 22:38:48

Scirius also allows you to search inside its database to find the elements you are looking for:

Screenshot from 2015-04-20 22:06:06

Scirius is able to handle multiple sources. So you can mixed local rules and rules download from outside sources such as Emerging Threats or SSLBL from abuse.ch:
Screenshot from 2015-04-20 22:05:52

Scirius is fetching activity information from Elasticsearch and it is now even able to display some interesting information about the state of your cluster
Screenshot from 2015-04-20 22:06:51

Scirius 1.0 is part of SELKS our live and installable Suricata NSM/IDS distribution. Happy SELKS can upgrade to scirius 1.0 via a simple apt-get update && apt-get upgrade. Other users can simply grab the release from Github.

The development will now focus on getting Scirius ready to handle IPS. So the changes will mostly be about rules transformation and the main features of Scirius should stay alike.

Stamus Networks is proud to announce the availability of the third release candidate of Scirius 1.0. This new release features minor bugfixes and an improvement.

Scirius 1.0-rc3 brings a new set of graphs that improve the visualization of the probe activity. The following video demonstrate the usage of this feature:



To sum up, the suricata page has now one zoomable sunburst graph who split the rules by classification:
sunburst-crop

There is also an alternate display via hierarchic circles:
circles-crop

The selection between the different type of graphs is done via the local settings:
Screenshot from 2015-04-03 09:07:31

You can download Scirius 1.0-rc3 from Github.

0

Stamus Networks is proud to announce the availability of SELKS 2.0 BETA1 release. With Jessie getting a target release date for 25 April 2015  SELKS will make the switch as well. The new Debian release Jessie will enable SELKS for much better HW compatibility, new kernel (3.16) and all the performance improvements, features and benefits with it right out of the box.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Please give it a try and do not hesitate to feed back!

SELKS2.0Beta1-1

 

 

Download SELKS 2.0 BETA1

More information: Howto and README

Follow us on Twitter, Google+ and Github

Get help at Freenode IRC on the #SELKS channel and/or  Google Mailing list.

 

0

Some words about PRscript

PRSCript is a script that run a series of builds and tests on a given branch. It was reserved to some developers so they can check the quality of their work before submission. The test builds are run on Suricata buildbot which is composed of some different dedicated hardware system. buildbot is an open-source framework for automating software build, test, and release processes. In the case of Suricata instance, it is set up to run various builds, unit tests as well as functional tests (such as pevma’s regression script).

The fact that this script was reserved to some users was a limitation as many contributors are not registered as Suricata buildbot users. As well, the fact that the code has to be public was not convenient as you could have to expose code before it is ready (with shameful TODO inside). Another point is that you were not able to customize your build. For instance, if you were introducing a new library as dependency it was not possible to test it before a global modification of the buildbot.

PRscript with docker support

To get over these limitations, Victor Julien and I have discussed on using Docker to allow developers to simply run a Suricata dedicated buildbot. As you may/should already know Docker is an open platform for distributed applications for developers and sysadmins. It allows you to quickly install install, manage and run containers. In our case, the idea was to start a pre-configured buildbot container using your local git as reference code. This way you can simply start test builds on your private code without even needing.

So, I have worked on this Docker based buildbot installation dedicated to Suricata and it has been merged in Suricata mainstream by Victor Julien.

It is now possible to use the prscript locally via Docker. Installation had been made simple so you should just have a few commands to run before being ready.

The buildbot will run various builds (gcc and clang, different builds options) and run suricata against some pcaps to check against possible crash.

Screenshot from 2015-04-07 16:22:19

Installation

Prerequisites

You need to have docker and python-docker installed on your system. Optionally you can install pynotify on your system to get desktop notification capability. On recent Debian based distribution you can use:

sudo apt-get install docker python-docker python-notify

Create the container

This operation has only to be done once. From the root of
Suricata sources, run:

sudo qa/prscript.py -C

It will take some times as the download is several hundred Mo. The result will be a docker container named ‘suri-buildbot’.

Using the buildbot

Start the buildbot

When you need to use the buildbot, you can start it from the command line:

sudo qa/prscript.py -s

You can check it is running via:

sudo docker ps

And you can connect to the buildbot web interface via http://localhost:8010

Start a build

Once the buildbot is active, you can start a build:

qa/prscript.py -d -l YOUR_BRANCH

This will start a build of the local branch YOUR_BRANCH without requiring any connectivity.

To get warned of the result of the builds via a desktop notification:

qa/prscript.py -d -l YOUR_BRANCH -n

Stop the buildbot

When you don’t need the buildbot anymore, you can stop it from the command line

 sudo qa/prscript.py -S

For further details, check Suricata docker QA page on OISF redmine.

Advanced usage

Build customisation

Buildbot will make suricata read all the pcap files available in qa/docker/pcaps/. So you can use this directory to add your own test pcaps.

Buildbot configuration is stored inside your suricata sources. It is the file qa/docker/buildbot.cfg. So, you can change the Buildbot configuration by editing this file. Then stop and start the docker container to get the new version used. This can be for example used when you need to add a flag to the configure command to activate a new feature.

What is great about this docker way of doing things is that it solves easily some complex points. For instance, if the buildbot configuration were coming from the Docker image then it will not be possible to easily edit it. Furthermore developer will loose any changes made in case of image upgrade. Also, the configure flags used by the buildbot will always be related to the current state of the code. So there will be no issue with running builds even if you are working on some older code as your buildbot configuration will be synchronized first.

Connect via ssh

The docker instance can be accessed via SSH using the admin account (password being ‘admin’ too). To get the port to use for ssh run the following command to get the port to use:

$ sudo docker port suri-buildbot
22/tcp -> 0.0.0.0:49156
8010/tcp -> 0.0.0.0:8010

and then connect:

ssh admin@localhost -p 49156

This can be used to install new dependencies inside the container. For instance if you are introducing a new library in Suricata, you may have to install the library in the docker instance.

Customizing the Docker image

On Docker side, the build recipes is available from GitHub. Feel free to modify it or propose updates and fixes.

0

Stamus Networks is proud to announce the availability of SELKS 1.2 stable release. SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

New features:

  • Suricata 2.1beta3  – Lua support for Stats output and Modbus parsing and matching as additional main features
  • Scirius 1.0-rc2 rule manager
  • Elasticsearch 1.4.3  – upgrade from 1.1.2
  • New Desktop icons – easy access to Dashboards and Scirius
  • Conkya free, light-weight system monitor for X, that displays any information on your desktop.”

 

system-status-scirius

Desktop-SELKS1.2

Desktop icons and Conky

You can download SELKS 1.2 from Stamus Networks’ open source page. Happy users of SELKS 1.1 can upgrade to SELKS 1.2 by using the traditional apt-get update && apt-get dist-upgrade. Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

NOTE – Elasticsearch upgrade for SELKS

If you were running Elasticsearch 1.1.2 with SELKS 1.1 this is the way to upgrade to Elasticsearch 1.4.3:

make sure your /etc/apt/sources.list.d/elasticsearch.list  looks like so

root@SELKS:~# cat /etc/apt/sources.list.d/elasticsearch.list
deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main
deb http://packages.elasticsearch.org/logstash/1.4/debian stable main

then run

apt-get update && apt-get dist-upgrade

Please make sure you consider some testing/verification for ES in a QA/test environment before doing the upgrade in the production environment.

Download SELKS 1.2

More information: Howto and README

Follow us on Twitter, Google+ and Github

Get help at Freenode IRC on the #SELKS channel and/or  Google Mailing list.

Stamus Networks is proud to announce the availability of the second release candidate of Scirius 1.0. This new release features bugfixes and improvements.

On the bugfix side, the main one is a fix in the display of graph that could fail on fresh install due to a problem in the Elasticsearch request
Screenshot from 2015-02-12 18:10:59

The system status has been improved to feature a warning phase on disk and memory usage.

The only new feature is the System Settings menu:

Screenshot from 2015-02-12 18:11:15

For now, it allows the administrator to setup two things:

  • HTTP proxy parameters: if activated and setup it will allow scirius to fetch rules updates using the specified proxy
  • Elasticsearch usage: some people are using scirius without Elasticsearch so displaying empty graph is not interesting for them. By unchecking elasticsearch, the graphs based on elasticsearch information are not displayed anymore.

Screenshot from 2015-02-12 18:05:54

You can download Scirius 1.0-rc2 from Github. SELKS users can upgrade to this release by doing apt-get update && apt-get dist-upgrade.

Stamus Networks is proud to announce the availability of version 1.0-rc1 of Scirius, our web interface for Suricata ruleset management. This new release is first 1.0 release candidate. You can download it from Github download page.

It features a lot of bug fixes and improvements over the previous (beta) release. Among the new features, Scirius is now displaying a system status in the left sidebar.

system-status-scirius

It displays :

  • Status of the Elasticsearch cluster (in SELKS and if setup).
  • Status of Suricata.
  • Memory usage: alerting if swap is used.
  • Disk status: alerting if disk is filled in.

An other important improvement is the support of flowbit, scirius now disables all rules sharing a flowbit if one is disabled. This helps preventing entering is some weird state where an incomplete set of rules could trigger a lot of events.

Last but not least, the copyright has been updated with a new year inside. Happy new year 2015 from Stamus Networks team.

SELKS user can upgrade to Scirius 1.0-rc1 via apt-get update && apt-get dist-upgrade.

0

Conky is a cool, desktop and lightweight monitoring tool. SELKS comes with a ready to use Conky config (also as part of the selks-scripts-stamus package).

With the installation of Conky – you get the possibility to do system monitoring on your desktop. Out of the box there are no Conky config files that are very useful for SELKS. That is why we created one. The trick is we used some info that Conky is capable of reading in itself for the system in general , however we added some stats that we harvest from the Suricata unix socket on the SELKS distro. That way you get right away  runtime, capture method , running mode and Suricata version right on your desktop – among other system stats like – Memory/CPU/Network usage.

The Conky config itself is already installed in /etc/conky/conky.conf however it is also present at /opt/selks/Scripts/Configs/Conky as part of the  selks-scripts-stamus  package for the purpose of record keeping(back up).

So if you are using the Desktop edition of SELKS, you can use Conky easily by running:

conky -d

in a terminal. Then you can close the terminal if you wish. That’s all 🙂

Desktop-SELKS1.2

The SELKS Conky config is best utilized when used with a screen resolution of 1680 x 1050 or more.

You can get further ideas for conky configs – just google “conky templates” there is plenty of stuff out there.

 

0

Introduction

Elasticsearch and Kibana are wonderful tools but as all tools you need to know their limits. This article will try to explain how you must be careful when reading data and explain how to improve this situation by using an existing Elastisearch feature.

The Problem

All did start with the analysis of an SSH bruteforce attack coming from Vietnam. This attack was interesting because of the announced SSH client “PuTTY-Local: Mar 19 2005 07:19:17” which really looks like a correct PuTTY software version when most attack don’t spoof their software version and reveal what they are using.

The Kibana dashboard was showing all information needed to get a good idea of attacks:

Screenshot from 2014-12-03 16:36:47

But when looking at less used and most used passwords, there was something really strange:

Screenshot from 2014-12-02 08:59:41

For example, webmaster is seen in the two panels with different values which is not logical.

By adding a filter on this value, the result was a bit surprising:

Screenshot from 2014-12-02 09:08:59

When looking at the detail of events, it was obvious this last result was correct. This SSH bruteforce has tried 10 different logins and has always used the same dictionary of 23 passwords.

To a solution

So the panels with top passwords and less seen passwords are displaying incorrect data in some circumstances. They have been setup in Kibana using the terms type.

This corresponds in Elasticsearch to a facets query. Here’s is the content of the query with the filter removed for readability:

{
 "facets": {
    "terms": {
      "terms": {
        "field": "password.raw",
        "size": 10,
        "order": "count",
        "exclude": []
      },
  }
}

So we have a simple request and it is not returning the correct data. The explanation of this problem can be found in Elasticsearch Issue #1305.

Adrien Grand is explaining that a algorithm returning possibly inaccurate values has been chosen to avoid a too memory intensive and network intensive search. The per-default algorithm is mainly wrong when they are more different values than searched values.

We can confirm that behavior in our case by asking for 30 values (on the 23 different passwords we have):

Screenshot from 2014-12-02 09:30:30

The result is correct this time.

If we continue reading Adrien Grand comment on the issue, we see that a shard_size parameter has been introduced to be able improve the algorithm accuracy.

So we can use this parameters to improve the accuracy of the queries. Patching this in Kibana is trivial:

diff --git a/src/vendor/elasticjs/elastic.js b/src/vendor/elasticjs/elastic.js
index ba9c8ee..8daa72a 100644
--- a/src/vendor/elasticjs/elastic.js
+++ b/src/vendor/elasticjs/elastic.js
@@ -3085,6 +3085,7 @@
         }
 
         facet[name].terms.size = facetSize;
+        facet[name].terms.shard_size = 10 * facetSize;
         return this;
       },

Here we just choose a far larger shard_size than the number of elements asked in the query. We could also have used the special value 0 (or Integer.MAX_VALUE) for shard_size to get perfect result. But in our test setup, Elasticsearch is failing to honor the request with this parameter. And furthermore, the result was already correct:

Screenshot from 2014-12-02 10:10:10

This patch has been proposed to Elasticsearch as PR 2106.

That was a small patch but this fixed our dashboard as the value in the terms panels are now correct:

Screenshot from 2014-12-03 16:44:57

0

Stamus Networks is proud to announce the availability of SELKS 1.1 stable release. SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

New features:

  • Suricata 2.1beta2 with SMTP support as main new feature
  • Optimized admin scripts
  • Scirius 1.0-beta1 rule manager
  • Authentication for remote access via HTTPS with user based role access
  • Improved Kibana dashboards and an addition of the SMTP dashboard
SMTP-Attachments

SELKS dashboard showing SMTP Attachments

Rule detail

Rule detail in scirius

You can download SELKS 1.1 from Stamus Networks’ open source page. Happy users of SELKS 1.0 can upgrade to SELKS 1.1 by using the traditional apt-get update && apt-get dist-upgrade. Please note that default login/password for HTTPS access is selks-user/selks-user.

More information: Howto and README

Follow us on Twitter, Google+ and Github

Get help at Freenode IRC on the #SELKS channel and/or  Google Mailing list.

Stamus Networks is proud to announce the availability of version 1.0-beta1 of Scirius, our web interface for Suricata ruleset management. This new release is a huge step toward 1.0 release as it contains a lot of new features and improvements. You can download it from Github download page.

The most visible update is the new design of the interface. It has been been completely changed thanks to Bootstrap CSS framework.

Screenshot from 2014-11-12 10:45:55

But the first change for user is that authentication and user management is now by default. Scirius is now multi user and features three level of permissions from read-only to superuser.

Another new feature is the display of graphics in some page. They are using Elasticsearch data. For example, the next screenshot is showing detail of a rule. A graph has been added to show the activity for that specific rules:

Screenshot from 2014-11-12 10:46:46

The interface is now more responsive as asynchronous requests are used to interact with Elasticsearch. This guarantee a responsive interface even if your Elasticsearch is slow.

SELKS user can upgrade to Scirius 1.0-beta1 via apt-get update && apt-get dist-upgrade. Please note that the default user/password on SELKS is selks-user/selks-user. Do not forget to change it after first login.

0

Stamus Networks supports its own generic and standard Debian Wheezy 64 bit packaging repositories for

These repositories provide Debian package for the newest Suricata IDS/IPS , htp releases and newest long-term stable kernel level version. SELKS already includes those repositories under /etc/apt/sources.list.d/selks.list.

You can use as follows:

wget -O – -q http://packages.stamus-networks.com/packages.stamus-networks.com.gpg.key | apt-key add – && \
apt-get update

Then  you can add the following :

deb http://packages.stamus-networks.com/debian/ wheezy main
deb http://packages.stamus-networks.com/debian-kernel/ wheezy main

in /etc/apt/sources.list.d/stamus.list for example.

The repositories contain packages for the long-term stable kernel level version. So if you would like to upgrade to the latest long-term supported kernel you can just do (on Debian):

apt-get update && apt-get upgrade
apt-get install linux-libc-dev linux-headers-3.14.19-stamus linux-image-3.14.19-stamus

 

UpgradeKernel

Kernel Packages

UpgradeKernel2

Kernel Upgrade

UpgradeKernel3

Verification

Those repos are included by default in SELKS.

Anther example:

apt-get install suricata

After giving a talk about malware detection and suricata, Eric Leblond gave a lightning talk to present SELKS at hack.lu conference.

Screenshot from 2014-10-23 13:46:02

You can download the slides here: 2014 hacklu selks

Introduction

SELKS 1.0 is featuring a privacy dashboard. This is a dashboard focusing on HTTP and TLS protocols. The used data source is events generated by Suricata for these two protocols. The goal of this dashboard is to show the different interaction between website. For example, you will see on the following video that opening elysee.fr which is the French president website is triggering the opening of page on Facebook and Google Analytics. This means that both Facebook and Google knows you’ve went to the presidential website.

Setup

The setup of the demonstration is simple as we are connecting to the web on the virtual machine. This has been done because it was easier to record the screencast in that case. But the most interesting setup consists in sniffing the traffic of the physical host from SELKS running on the virtual machine. This way, SELKS will analyse your local traffic and you will be able to see in SELKS all the events coming from your real internet life.

The setup is simple. In Virtualbox, go to the machine details and click on network. Then choose to bridge your physical network interface and allow promiscuous mode on the interface:

Screenshot from 2014-10-19 12:10:43

Demonstration

Watch the following video to discover how this dashboard can be used:

An other way to use this privacy dashboard is to use one of the filter. For instance, if we filter on http.http_refer:"http://www.whitehouse.gov" we get a dashboard containing all HTTP events with a referrer being the US president website. So if you look at the hostname on the following screenshot, you will see that going on whitehouse.gov also lead you to external websites

Whitehouse links

My favorite in this list is www.youtube-nocookie.com but something like cloud.typography.com is really interesting too. Even a website like whitehouse.gov is not anymore hosting is own fonts.

The privacy dashboard is also containing TLS information extracted by Suricata. It lists TLS connections done on well know wesbite such as Facebook, Twitter or Google. For example, we can see that going on CNN cause some TLS hits on Twitter and Facebook.
Screenshot from 2014-10-19 12:00:45
TLS being encrypted we can’t prove this link and that’s the short time frame that stand for a proof of the link between websites.

Conclusion

SELKS privacy dashboard is just an example of what you can achieve in SELKS by using Suricata network security monitoring capabilities. The demonstration shown here is local but don’t forget you can do it at the level of a whole network.

0

Stamus Networks is proud to announce the availability of SELKS 1.0 stable release. SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Screenshot from 2014-10-15 21:39:11
You can download SELKS from SELKS main page.

SELKS is comprised of the following major components:

It offers proven, powerful, innovative and scalable open source multi-threading technologies in a bundle.

SELKS 1.0 comes with 10 pre-installed Kibana IDS/NSM dashboards. They cover analysis of the Suricata alerts and events with per-protocol dashboards (Alerts, HTTP, Flow, SSH, TLS,DNS …). Some dashboards are also dedicated to more specific tasks – like the PRIVACY dashboard:
Screenshot from 2014-10-15 21:28:27
It shows privacy related information such as which page are leading to well know personal data providers such as Facebook, Twitter or Google.

SELKS provides Scirius – a rules management interface for Suricata. Scirius has been developed by Stamus Networks to provide interaction with Kibana and Elasticsearch. It displays for example statistics on rules and links to existing Kibana dashboards:
Screenshot from 2014-10-15 21:17:37

Scirius provides up-to-date signatures via EmergingThreats Open (or PRO ) ruleset and SSL abuse.ch signatures
Screenshot from 2014-10-15 21:18:29

Scirius can be upgraded via standard Debian method (apt-get upgrade). Stamus Networks is also determined to provide the latest stable Debian kernel release for SELKS. Upgrade to the latest stable kernel is easy via the package system. For example, it is possible for the user running the installed version to upgrade the kernel to the latest 3.14 version:

kernel-upgrade-3.14.21
Scirius 1.0rc1 can upgrade to the 1.0 version by running apt-get dist-upgrade

The list of provided Kibana dashboards will be augmented in the future and this will be done seamlessly via the Debian packaging system and Kibana autodiscovery:

Kibana-dashboards

We really hope you will enjoy SELKS  an enterprise-grade IDS and Network Security Monitoring system in 30 seconds.

How to and README

Follow us on Twitter, Google+ and Github

Lets talk about SELKS…

0

Stamus Networks is proud to announce the availability of SELKS 1.0 RC1. This is the first release candidate of our live and installable ISO based on Debian implementing a ready to use Suricata IDS/IPS. More about SELKS you could read on our Open Source page.

This release includes major overhaul and improvements:

  • Introducing for the first time the new Stamus Networks package repositories developed especially for SELKS – Kibana, Scirius
  • Update and upgrade all software and SELKS the Debian way (apt-get or aptitude)
  • 9 ready to use out of the box IDS/IPS dashboards
  • Over 150 fields to search,select,filter and easily analyze upon right out of the box
  • Fully enabled logging
  • Suricata 2.1beta1 (adding flow and alert payload logging to the NSM arsenal)
  • Scirius 0.8  (latest release of our graphic Suricata ruleset manager)

A better interface

SELKS 1.0 RC1 comes with preloaded dashboards and a modified version of Kibana:

Dashboards

Screenshot from 2014-09-09 20:44:42
This allows interaction with Scirius, our open-source Suricata ruleset management interface:

Screenshot from 2014-09-09 20:26:15

SELKS 1.0 RC1 contains Suricata 2.1beta1 which brings flow and alert payload logging – available right out of the box on the predefined dashboards:

Screenshot from 2014-09-09 22:45:00

Alert-SELKS-Payload1

Easy upgrade

Stamus is dedicated to provide the latest releases of Suricata, htp and kernel level. That’s why we provide generic Debian packaging for the newest Suricata IDS/IPS , htp releases and newest long-term kernel level version (3.14.18 at the time of this writing).

SELKS comes with a standard Debian Wheezy distribution with 3.2 kernel – if you would like to upgrade to the latest long-term supported kernel you can just do (for example kernel 3.14.18):

apt-get update && apt-get upgrade
apt-get install linux-headers-3.14.18-stamus linux-image-3.14.18-stamus

For everything else you can just do:

apt-get update && apt-get upgrade

As easy as that!

DOWNLOAD SELKS HERE

 

Stamus Networks is proud to announce the availability of the version 0.8 of Scirius, the web management interface for Suricata. This new release contains a lot of new features as well as bug fixes.

On the functional side, the main new features are:

  • Support for content such as IP reputation list
  • Changelog support: display change on sources after update
  • Global search: text search in all objects
  • The changelog on source is really useful to know what signatures have been added or modified:
    Screenshot from 2014-09-03 16:51:18

    The global search is accessible from the top bar in all pages. It allows you to quickly access to the matching objects:
    Screenshot from 2014-09-03 16:53:23

    Among the other features, one can also mention the syntax highlighting for the rule. Rule detail now comes with information about rule status in rulesets and rule stats:
    Screenshot from 2014-09-03 16:36:58

    We hope you will enjoy this new release. As usual it can be downloaded from Github. Happy NIDSing!

Thanks to the EVE JSON events and alerts format that appear in Suricata 2.0, it is now easy to import Suricata generated data into a running Splunk.

To ease the first steps of integration, Stamus Networks is providing a Splunk application: Suricata by Stamus Networks

It can be installed like any other applications and it just requires that a Suricata EVE JSON file is known and parsed by Splunk.

Current version is providing a dashboard and a few searches:

Screenshot from 2014-07-30 15:39:11

This post describes how to import the application and if you don’t have already done it how to import data from a Suricata EVE file.

Importing the application

Importing the application is done via the Apps menu on top of Splunk starting page:

Screenshot from 2014-07-30 15:33:39

Suricata by Stamus Networks application is currently provided as a file, so you need to download it: Suricata by Stamus Networks. Once done, you can add the application:

Screenshot from 2014-07-30 15:33:50

You need to select the file

Screenshot from 2014-07-30 15:34:05

Importing a Suricata EVE JSON file

Since splunk 6.1.x, the recognition of the file format is automatic. If you are using an older version of Splunk, you may need to refer to this page to import Suricata EVE file.

Here’s the detailed procedure to import Suricata EVE data into Splunk. From the starting page, we click on Add Data:

Screenshot from 2014-07-30 15:27:48

Then we click an Files & Directories to tell Splunk to import data from Suricata EVE JSON file:

Screenshot from 2014-07-30 15:28:08

Once done, we click on the New button:

Screenshot from 2014-07-30 15:28:21

Now, we only need to give the complete path to the eve.json file:

Screenshot from 2014-07-30 15:28:47

Once this is done, we just need to click on all Continue buttons to be done.

Using the application

Now, we can go to the application by clicking on Suricata by Stamus Networks:

Screenshot from 2014-07-30 15:34:42

Next step can be to to go the dashboard:

Screenshot from 2014-07-30 15:35:02

The dashboard contains some interesting panels like the following one who displays the destination IP addresses that are using a self-signed certificate for TLS connections:
Screenshot from 2014-07-30 14:37:52

Conclusion

This application should evolve with time, so stay tuned and follow us on twitter for more information.

Stamus Networks is proud to announce the release of SELKS 1.0 beta2. This is the second public release of our Live and installable ISO implementing a ready to use Suricata IDS/IPS.

Screenshot from 2014-05-22 10:14:38

SELKS 1.0 beta2 can be downloaded:

MD5 sum of the SELKS-1.0beta2.iso file is 38222aeda399f7502913c91465ac9499.

If this new release features some improvements in the creation process, the main new things for the user are an updated version of Scirius and a custom Kibana interface. A menu to switch from one interface to the other has been added on both application. A link has been added in the detail of alert event to be able to jump from Kibana to the correct place in Scirius rule management. The following screencast demonstrates these features:

On Suricata side, file extraction and Unix socket are now enable by default. So SELKS 1.0-beta2 will extract to disk files from stream if signatures containing the filestore are used. The activation of Unix socket allows user to get data from Suricata and/or to use alternate running modes like multiple pcap processing.

The complete Changelog is as follows:

  • bump ES to 1.2.1
  • suricata: enable file extraction
  • kibana: use stamus version
  • suricata: enable unix-socket
  • scirius: remove unused files
  • build: add capability to add option to lb config
  • scirius: use new command to build default ruleset (Fix Issue 1)
  • scirius: use version 0.4
  • doc: update links on desktop README
0

The Ubuntu used in this tutorial:

root@LTS-64-1:~/opt#uname -a
Linux LTS-64-1 3.5.0-45-generic #68~precise1-Ubuntu SMP Wed Dec 4 16:18:46 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

If you have these packages installed you need to remove them so that Scirius would work with the latest python dependencies.
Please be careful so that this actually does not affect your current running services. It is always best to test first 🙂

root@LTS-64-1:~/opt#apt-get remove django-tables python-django python-django-south python-git

Install the needed dependencies:

root@LTS-64-1:~/opt#aptitude install python-pip git
root@LTS-64-1:~/opt#pip install django django-tables2 South GitPython pyinotify daemon

Clone the latest version

root@LTS-64-1:~/opt#git clone https://github.com/StamusNetworks/scirius.git
root@LTS-64-1:~/opt#cd scirius/
root@LTS-64-1:~/opt/scirius# python manage.py syncdb

Start Scirius

root@LTS-64-1:~/opt/scirius#python manage.py runserver
Validating models…
Failed to setup thread-interrupt handler. This is usually not critical
0 errors found
May 20, 2014 – 19:51:27
Django version 1.6.4, using settings ‘scirius.settings’
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

If you need to connect to the server remotely (provide your ip)  –

root@LTS-64-1:~/opt/scirius#python manage.py runserver 10.0.10.5:8000
Validating models…
Failed to setup thread-interrupt handler. This is usually not critical
0 errors found
May 20, 2014 – 19:51:58
Django version 1.6.4, using settings ‘scirius.settings’
Starting development server at http://10.0.10.5:8000/
Quit the server with CONTROL-C.

Now lets have a walk through registering and adding a ruleset

For example (for the latest stable and dev Suricata) from  http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz since Emerging Threats create, write and distribute specially tuned for Suricata rulesets that make  use of the advanced features of that IDS engine.

First we need to add a source:AddRuleset-1

AddRuleset-0

 

Then we add a ruleset:

 

AddRuleset-2

We need to edit the ruleset and select the categories we want from that ruleset:

 

AddRuleset-3

 

AddRuleset-4

Select categories:

AddRuleset-5

Validate changes:

AddRuleset-6
If you have already Elasticsearch, Logstash, Kibana installed on the same server,
you could do the following -> put in the values as on the picture – except the host name – chose the hostname to be exactly as your hostkey in Elasticsearch/Kibana , like in the pics below.

 

hostkey1 hostkey2

 

Then in Scirius ->

CreatingSuricata-1 CreatingSuricata-2

 

Now you should be able to see the hits and which rules are making the most noise 🙂

That’s it for a quick intro.

 

Stamus Networks is proud to announce the release 0.3 of Scirius, our web interface for Suricata ruleset management.

The interface has been redesigned for more compacity and clarity:
Screenshot from 2014-05-19 11:21:00

Two major features have been added:

  • Support of local rules: User can now upload rules contained in an archive
  • Fast suppression of rules: two clicks are enough to suppress one rule

It is now also possible to select the time period selection on rules activity:

Screenshot from 2014-05-19 11:28:07

 

Please note, the rules with sid 220029 on the screenshot. It is displayed strikethrough because it has been suppressed from the ruleset.

Here’s a screencast showing how easy it is to suppress a noisy rule from a ruleset:

With all these new features, we think that Scirius can now be efficiently used to administrate a Suricata ruleset.

Stamus Networks is happy to release Scirius as Open Source Software under GPLv3. You can download it from GitHub : scirius-0.3.tar.gz.

 

 

Stamus Networks is proud to announce the first release of Scirius, its Suricata ruleset web management interface.

Scirius is a web management interface developed by Stamus Networks and released under the GPLv3 license. The interface is aiming simplicity and efficiency and that’s why we have adopted a simple design:

Screenshot from 2014-05-03 11:25:06

It is possible to link Scirius with a running Elasticsearch fed by Suricata EVE JSON log. Once done, information stored in the Elasticsearch can be used to get an idea of the activity of the Suricata. The following screenshot is an example of statistics fetched from Elasticsearch and displayed in Scirius:

Rules activity

Scirius is currently in alpha stage but it is already possible to manage efficiently a Suricata ruleset using ETOpen or ETPro ruleset. For example, the following video is demonstrating how it is possible to remove a selected subset of signatures from the ruleset:

Scirius is available on Github. Following releases of Scirius will feature among other things the support for local signatures (uploaded by the user) and some missing operations such as quick removal of individual signature.

I’ve given a talk entitled “Suricata 2.0, Netfilter and the PRC” at the Hackito Ergo Sum conference.

The talk is presenting Suricata and the new features available in version 2.0, focusing on the new EVE output and how it can be used with Elasticsearch, Logstash and Kibana. I’ve also shown how ulogd, the Netfilter logging daemon can be used with Elasticsearch thanks to the new JSON output plugin. Finally, I’ve explained how I’ve discovered a attack schema which is originating from systems running in the People Republic of China.

You can get the slides here: Suricata 2.0, Netfilter and the PRC

This is the first blog post on Stamus Networks technical blog. You will find here posts focused on Intrusion Detection System and Network Security Monitoring as well as information specific to Suricata or our products.