« May 2010 | Main | January 2010 »

Sunday, March 21, 2010

Service Monitoring with the TEMPer USB thermistor

In the first part of this tutorial, we looked at the perl modules for the TEMPer device. I also wanted a way to use this for monitoring and logging, and as I don't happen to be proficient in perl, I decided to write a bash script that will call the perl script. In this part, we'll look at the bash script, a php program that can act as a central logger or notification tool, and also how to use the bash script as a plugin for nagios. Please keep in mind that this document is written for Unbuntu 9.10, this may also work with other operating systems and versions, but this isn't tested elsewhere. Also, please note that this is a revised version, I've made some significant improvements, I think.

Bash: check_temper

The bash script is named check_temper (to mesh with nagios's naming convention, and is listed here. First off, download the bash script and check the script:

wget -O check_temper http://www.cs.unc.edu/~hays/dev/bash/temper/check_temper
chmod a+x check_temper
sudo ./check_temper 

You'll likely get an error like this:

/usr/local/bin/temper_mon.pl not found

check_temper looks for the perl script, temper_mon.pl in /usr/local/bin/, which is the standard local for user installed software on most linux systems. In the first part of this, we left the temper_mon.pl file in the home dir. It would be better to have check_temper in /usr/local/bin/ also, so let's move that:

sudo cp temper_mon.pl /usr/local/bin

Now try the check_temper file again:

sudo ./check_temper 

You should get back something like:

Critical: Temperature is 78.8 F

If you take a look at the script, you'll see that there are some variables at the top that define what a warning temperature is, and what a critical temperature is. You can also control whether output is in fahrenheit or celcius.

If you run the program again, and then check the error code, you should see that a critical temp reports an error code of 2, a warning an error code of 1 (2 is also used if something is not found or didn't work), and 0 if the temp is low enough. For example:

bil@test:~$ sudo ./check_temper 
Critical: Temperature is 78.8 F
bil@test:~$ echo $?
2

Now copy check_temper to /usr/local/bin so it's available on the path:

sudo mv check_temper /usr/local/bin

The way the script works, it looks at the output from the temper_mon.pl script, and compares the temperature to the warning and critical values. If the temperature is below the warning level, the program exits with a 0 error code. Anything between the two generates a 1 error code, and if the temperature is above the critical level, the program exits with a 2 error code. This behavior is consistent with nagios as we will see, but it also allows you to use this script with other programs.

PHP Monitor

If you look in the variable section of check_temper, you'll see that you can specify a url. The idea here is that the script can pass data to a program on a web server, which will then do some logging or send an alert. The bash script uses wget to GET a web page from the server and passes variables to the server, specifically the temperature, error code, and the message. If you don't want to use this feature, you can just leave that line commented out.

I've written a pretty simple php program that can process the data passed in the GET. What it does is take in the data, and then writes out a log file with the ip address of the machine that send the get, with a time and data stamp, the ip address, the temperature, the error code, and the message. I'm not going to tell you how to set up a php server, but it's not that hard to do. But when you do, do please make sure to restrict access to it if you use this program, it doesn't do much sanitizing of user data, and since it does send email, I'm not going to claim that it's secure. All we want here is for a machine to be able to report to a server, so restricting access by password and ip numbers should be sufficient. You can download a zip of the php directory I use. The package includes an emailer function I use, and a file directory where the logs get written--you need to make sure the web server can write to that directory.

If the temperature is too high, it will also write out two additionals files, a WARNING file that acts as a log of all the warnings seen for that IP address, and an ALERT file that acts as a marker for when a warning was sent. If the ALERT file isn't stale, no email is sent (this allows one to limit the number of warnings sent via email.) The interval that defines staleness is in the variable section of the php program. There's also a default high temperature setting, and a case statement that allows one to tailor temperatures for other devices.

Now, this php program is a bit unusual--it does not put out any html. There are some echos and whatnot that are commented out, you can uncomment these and call the page with a get statement to test things out to get it working. But the program is really just designed to act as a logger/warning. Of course, at some point I'll write a php program to read the data being logged, but that's a job for laterman. What the program does do is to write out a file in the files directory with the ip number of the host reporting as the file name, and the data (timestamp, ip, temp, error code and message) in tab delimited format.

So, if you want to use just this php program as your monitor, that should work fine--what you'd do is set up a cron or an upstart to call this however often you want it to run. If you're new at this stuff, the cron's the easier way to go.

Nagios

The way the bash program is written, it can also serve as a nagios plugin via nrpe. This will not do you any good unless you already have a nagios server setup and running. If you don't know what I'm talking about, you can either run off and learn nagios, or just skip this part. Nagios isn't trivial, but I'll wait here until you come back.

To use this, first install the nrpe server for nagios:

sudo apt-get install nagios-nrpe-server
On a system that's bound to an external NIS database for users and groups I got the following error:
Unpacking nagios-nrpe-server (from .../nagios-nrpe-server_2.12-3ubuntu1_i386.deb) ...
addgroup: The group `nagios' already exists as a system group. Exiting.
usermod: user 'nagios' does not exist in /etc/passwd

I imagine that Any system bound to an external user system such as LDAP might get a similar message, this is a known bug. So I added:

nagios:x:114:121::/var/lib/nagios:/bin/false

as the last line in /etc/passwd, since that's the line that this installation puts into that file on a standalone machine. That let me run the installation, and afterwards, I just removed that line. I don't think this will hurt anything, but if you know better, please let me know. You won't need to do this on a stand along machine, the nrpe server install will create the nagios account for you.

To make things easier to troubleshoot, we'll run our nagios plugin as a user named nrpe. The reason for this is that the nagios nrpe debugging information is a bit sparse, and for troubleshooting, it's handy to be able to login to the same account as nagios will use to run the plugin.

sudo adduser nrpe

Next link the check_temper file to the nagios plugins directory:

sudo ln -s /usr/local/bin/check_temper /usr/lib/nagios/plugins/ 

Make sure to double check the permissions to make sure it's only writable by root. Now, edit the /etc/nagios/nrpe_local.cfg file:

sudo vi /etc/nagios/nrpe_local.cfg

and add this line (and please note this is a change from the previous verson of this article, I found that using the sudo command prefix in the nrpe.cfg file interferes with other plugins):

command[check_temper]=sudo /usr/lib/nagios/plugins/check_temper

This tells the nrpe service to run /usr/lib/nagios/plugins/check_temper when nagios contacts it and asks for check_temper. Please note, you can also make or use a directory, /etc/nagios/nrpe.d and put your config file there--if there are any files that are in that directory, nrpe will try to use them. This is useful if you're adminning multiple machines with similar configurations.

Next, edit the config file /etc/nagios/nrpe.cfg

sudo vi /etc/nagios/nrpe.cfg

Replace with the IP address of localhost that is running the Nagios monitoring server:

#allowed_hosts=127.0.0.1
allowed_hosts=your.nagios.server.address 
Then comment out the nrpe=nagios line and add a line making the nrpe user to nrpe:
#nrpe_user=nagios
nrpe_user=nrpe
And then set the group:
#nrpe_group=nagios
nrpe_group=nrpe

Also, uncomment the line command_prefix=/usr/bin/sudo Don't do this after all, it's easier to use the sudo command in the .cfg file.

# This lets the nagios user run all commands in that directory (and only them)
# without asking for a password.  If you do this, make sure you don't give
# random users write access to that directory or its contents!
#command_prefix=/usr/bin/sudo 

If you running nagios and this temperature monitor on the same machine you can uncomment the server_address=127.0.0.1 line:

Finally, check the bottom of the file to make sure that the include for the nrpe_local.cfg is enabled:

# local configuration:
#       if you'd prefer, you can instead place directives here
include=/etc/nagios/nrpe_local.cfg

Next, open the /etc/nagios/nrpe_local.cfg file:

sudo vi /etc/nagios/nrpe_local.cfg
And add the line telling nrpe where check_temper is located:
command[check_temper]=/usr/lib/nagios/plugins/check_temper

Then save the file and restart the nrpe service:

sudo /etc/init.d/nagios-nrpe-server restart

If you're new to nagios, you might look into other commands that work through npre. For example, you can use npre to check load:

command[check_light_load]=/usr/lib/nagios/plugins/check_load -w 6.00,6.00,6.00 -c 10.00,10.00,10.00

or look for zombies:

command[check_zombies]=/usr/lib/nagios/plugins/check_procs -s Z -w 3 -c 6

Next, softlink the check_temper file to the /usr/lib/nagios/plugins dir:

sudo ln -s /usr/local/bin/check_temper /usr/lib/nagios/plugins

So far so good. Open a ssh connection to your nagios server. If you don't have nrpe support, you need to install it:

sudo apt-get install nagios-nrpe-plugin

Now try running the check_nrpe plugin from the nagios server, pointing it at your temperature monitor:

/usr/lib/nagios/plugins/check_nrpe -H test -c check_temper

You should get something like this:

-sh-3.2$ /usr/lib/nagios/plugins/check_nrpe -H test -c check_temper
check_temper failed to log temp F

That's ok, we know from this that the check_temper program got called, and somehow failed.

The reason it didn't report back a temperature is probably because the check_temper isn't allowed to run in privileged mode, and thus cannot query the device.

If, on the other hand, you get:

CHECK_NRPE: Error - Could not complete SSL handshake.

that probably means you didn't restart the nrpe service on your remote system, or didn't add your nagios server as an allowed host.

Or, if you get:

NRPE: Command 'check_temper' not defined

it's probably a problem with the line in the nrpe_local.cfg file.

What we need to do now is added nrpe to the sudoers file so that it can run commands without a password. To do this we run the visudo command:

sudo visudo -f /etc/sudoers

Add this line to the file at the bottom and try it out:

nrpe  ALL=(ALL) NOPASSWD: /usr/lib/nagios/plugins/

Now, log in as nrpe, and try to run the plugin with out a password:

sudo /usr/lib/nagios/plugins/check_temper

It should run without you having to provide a password. Should's a big word though. If it asks for a password, double check your /etc/sudoers file.

Now, go back to your nagios server and try calling the plugin again:

/usr/lib/nagios/plugins/check_nrpe -H test -c check_temper

Hopefully, this time you got good output.

Now, what you should probably do is go back to the /etc/nagios/nrpe.cfg file and change the user and group from nrpe to nagios, so that your installation is in line with a standard installation. If you do this, you'll also need to use visudo to edit the /etc/sudoers file removing nrpe's privileges and grant the same priviledges to nagios. But if you don't want to bother, at least disable nrpe's ability to login and use a shell, so as to safeguard your system.

At this point all you need to do is add this service into the service.cfg and host.cfg files in /etc/nagios.

Example of host.cfg entry:

define host{
use                     generic-server
host_name               test.unc.edu
alias                   TEST
address                 test.unc.edu
}

Example of services.cfg entry:

define service{
use                             generic-service
host_name                       test.unc.edu
service_description             CHECK-TEMPERATURE
contact_groups                  server-admins
check_command                   check_nrpe!check_temper
}

So there you have it, a simple set of ways to leverage the USB TEMPer module to set up an alert system. Please let me know if you run into any problem with this, and best of luck.

Posted by bil at 4:38 PM
Edited on: Saturday, May 08, 2010 11:15 AM
Categories: My Software, Other Software, Work
Comment by Joseph Dyland - Thursday 29th April 2010 11:40:15 PM

Great post, Very helpful and informative, I had purchased two of these units and was at a standstill for quite a while.

Would you happen to have any tips on getting the temperature to populate the performance data field in Nagios?, I would love to be able to graph temperature overtime and am so close now!
Comment by bil - Friday 30th April 2010 08:36:53 AM

I'll look into it, thanks,
Comment by Anonymous Coward - Friday 30th April 2010 08:51:36 AM

I ended up figuring it out, I found that performance data is taken from any thing after |

So I simply added a pipe to the area where the message is being printed out, adding temperature= then the variable of the temperature it self, followed by minimum and maximum values, It is now working exactly as I have wanted!

message="${message} Sensor ${iteration} Temp is ${x} ${temp_ind}|temperature=${x};55;60"

In nagios it looks like this,

Status Information: Normal: Sensor 0 Temp is 25.5 C
Performance Data: temperature=25.5;55;60

Now I do have one more question, Do you know how I could add a second TEMPer sensor using the methods you have provided?

Comment by bil - Friday 30th April 2010 09:41:45 AM

I haven't written up the docs yet, but I updated the check_temper bash script to support multiple sensors. So if you download it again, it should just work with 2-3 sensors.
Comment by Riccardo - Wednesday 08th September 2010 09:48:22 AM

Great Post !
Only a note, the PHP directory structure zip file is not more available !

Bye
Riccardo
Comment by bil - Thursday 16th September 2010 01:35:27 PM

Thanks for letting me know, it's up now. Expect a new version in a few weeks.. ..

Themes for Thingamacomment

I've started putting together some themes for Thingamablog that support the Thingamacomment software. Only the template files are modified, the stylesheets have been left alone since I'm not very good with CSS yet. To make it easy to keep these separate from exisiting themes, I've adopted a naming convention of adding "TC" to the end of these--you can safely install them using the Tools, Install Theme Pack, and then apply that theme to your blog. The ones I've done so far are up at https://wwwx.cs.unc.edu/~hays/dev/php/thingamacomment_themes/ 

Posted by bil at 2:24 PM
Edited on: Sunday, March 21, 2010 3:49 PM
Categories: My Software

Tuesday, March 16, 2010

Installing Snort and Barnyard2 in Ubuntu 9.10: Part 1

This is based on Nick Moore's Snort_2.8.4.1_Ubuntu.pdf. There are some significant differences, but I'm following his lead. This is also useful:
http://ubuntuforums.org/showthread.php?t=145641

I'm assuming you have a working installation of Ubuntu 9.x, either in a virtual machine or on real hardware. Even if you have real hardware, you might consider using Sun's Virtualbox to do a test machine--you can do a base install, then take a snapshot here and there before throwing yourself into this. That way if things go south you can back out to the snapshot cleanly. Also, keep in mind, I don't even pretend to understand all of this myself, it's just a document I put together to document what I got to work for me.

Background

If you're interested in installing snort you probably already know something about it. But in case you don't:
  • Typically, you install snort to listen on a second network interface, so you can access your machine via the primary, and have snort listen on a second interface. Usually this second interface doesn't have an ip number.
  • You don't have to do that tho, and for a test install like this one, it's fine to just use the primary interface. In my case, my primary is eth1 and that's where everything is pointed. If you want to use a different interface, you can, just change eth1 to eth0 or eth2 or whatever. The ifconfig command will show you what you have available.
  • Snort does the analysis of packets. To take load off of snort, we'll use barnyard2 to handle logging of traffic. The way this works is snort logs what it finds in a snort.log file, and barnyard tails that file and puts the information into the mysql database.
  • We'll also install base and adodb, these packages let us put a pretty web interface on the snort data.

Installing some unixy bits

First thing we need to do is install some packages. Open a terminal window. If you don't know what a terminal window is, please go away until you do. From the command line run this (you can cut and paste this if you like, but review the command before executing it):
sudo apt-get install libpcap0.8-dev libmysqlclient15-dev mysql-client-5.0 \
mysql-server-5.0 bison flex apache2 php5 libapache2-mod-php5 php5-gd php5-mysql \
libtool libpcre3-dev php-pear vim ssh openssh-server
As part of the MySQL install, it will prompt you for a root password for that. I suggest you choose one password for this and all of the other admin passwords you set up throughout this process, it will reduce your confusion. This will take a while to run.

Set up Snort and MySQL

First, we install the mysql version of snort:
sudo apt-get install snort-mysql
This will prompt you for which interface you want to use, and also ask you about your home network. Don't worry about the home net setting, we'll twiddle that later. Do make sure to let it set up a database for you.

Next, we have to make a database for snort to use (before we run the commands the above install suggested!). Start mysql thusly:
mysql -u root -p
Then at the mysql prompt enter the following:
create database snort;
grant CREATE, INSERT, SELECT, DELETE, UPDATE on snort.* to snort@localhost;
SET PASSWORD FOR snort@localhost=PASSWORD('YOURPASSWORDHERE!');
exit
Next, move to the directory where a template for the database is stored
cd /usr/share/doc/snort-mysql/
zcat create_mysql.gz | mysql -u root -p snort
This will import a schema for the snort db into mysql. Now we will check to see that the Snort database has been correctly installed. Start mysql again:
mysql -u root -p
Then at the mysql prompt:
SHOW DATABASES;
use snort;
SHOW TABLES;

You should see something like this:
mysql> SHOW DATABASES;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| snort |
+--------------------+
3 rows in set (0.00 sec)

mysql> use snort;
Database changed
mysql> SHOW TABLES; +------------------+ | Tables_in_snort | +------------------+ | data | | detail | | encoding | | event | | icmphdr | | iphdr | | opt | | reference | | reference_system | | schema | | sensor | | sig_class | | sig_reference | | signature | | tcphdr | | udphdr | +------------------+
16 rows in set (0.00 sec)
mysql>
Type exit, and we're done with that

Now, we need to edit the snort.conf file, in /etc/snort:
sudo vim /etc/snort/snort.conf
Find "var HOME_NET any", and comment it out and add a line specifying the interface you want to use:
#var HOME_NET any
var HOME_NET $eth1_ADDRESS
Find "output log_tcpdump: tcpdump.log" and comment that out:
#output log_tcpdump: tcpdump.log
Find "output log_unified". Insert the following below it:
output unified2: filename snort.log, limit 128
Now a quick test. Run the following and see if snort runs:
sudo snort -c /etc/snort/snort.conf -i eth1
If you get to this:
--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.8.4.1 (Build 38)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
Copyright (C) 1998-2009 Sourcefire, Inc., et al.
Using PCRE version: 7.8 2008-09-05

Rules Engine: SF_SNORT_DETECTION_ENGINE Version 1.10 <Build 16>
Preprocessor Object: SF_SSH Version 1.1 <Build 1>
Preprocessor Object: SF_DCERPC Version 1.1 <Build 4>
Preprocessor Object: SF_DCERPC2 Version 1.0 <Build 1>
Preprocessor Object: SF_SMTP Version 1.1 <Build 7>
Preprocessor Object: SF_FTPTELNET Version 1.2 <Build 11>
Preprocessor Object: SF_Dynamic_Example_Preprocessor Version 1.0 <Build 1>
Preprocessor Object: SF_DNS Version 1.1 <Build 2>
Preprocessor Object: SF_SSLPP Version 1.1 <Build 2>
Not Using PCAP_FRAMES

you've successfully installed snort!

Setting up BASE and Adodb

In this step, we will set up the web environment for BASE and Adodb. First thing we need to do install some support for mail and smtp with pear.
sudo pear install --alldeps Mail
sudo pear install --alldeps Mail_Mime
Then in your home directory, make a space to keep the files we'll be working from as we build this:
cd
mkdir snortinstall
cd snortinstall

At this time the current version of BASE is 1.4.3.1 and you can get it from the Base site on Sourceforge:
wget -O base-1.4.4.tar.gz \
http://sourceforge.net/projects/secureideas/files/BASE/base-1.4.4/base-1.4.4.tar.gz/download
Get adodb4991.tar.gz from Adodb's site on Sourceforge:
wget -O adodb4991.tgz \
http://sourceforge.net/projects/adodb/files/adodb-php-4-and-5/\
adodb-4991-for-php/adodb4991.tgz/download


Next, we'll unpack adodb and base, these are collections of php pages used to interface a web page to the mysql database into which we'll be feeding snort data.

cd ~/snortinstall
tar -xzvf adodb4991.tgz
tar -xzvf base-1.4.4.tar.gz
sudo mv adodb /var/www
sudo mv base-1.4.4 /var/www
Open the php.ini file:
sudo vim /etc/php5/apache2/php.ini

Find “Dynamic Extensions" and add the following lines at the bottom of that section:
extension=mysql.so
extension=gd.so

Now a minor edit of the httpd server config:
sudo vim /etc/apache2/apache2.conf
At the bottom of the file, insert the line
servername your.server.name.domain
Exit and then restart the web server:
sudo /etc/init.d/apache2 restart
Next, create a softlink to the base directory, this way you can install an update and just move the link. Also, we'll make the base dir writable so the webserver can update the config when we're done. I wouldn't do this on a multi-user machine tho:
cd /var/www
sudo ln -s base-1.4.4 ./base
chmod  a+w base

In a browser, go to http://localhost/base, you should be presented with the base configuration page. We'll walk through this and create additional links and mysql entries. Click “continue”
Step 1
Set the path to adodb to /var/www/adodb
Step 2
Database Name=snort
Database Host=localhost
Database User=snort,
Database Password=yourpassword
Step 3
check use authentication system
Admin User Name=snort
Password=yourpassword
Full Name=snort
Step 4
Click “Create BASE AG”
Step 5
Test your login and password.
Then, very important, reset the permissions on the base dir:
chmod og-w base

Setting up Barnyard

Barnyard was written to take over the various output processing tasks to take some load off of snort. We'll be using barnyard2.
cd ~/snortinstall
wget -O barnyard2-1.7.tar.gz \
http://www.securixlive.com/download/barnyard2/barnyard2-1.7.tar.gz
tar zxvf barnyard2-1.7.tar.gz cd barnyard2-1.7 ./configure --with-mysql make sudo make install sudo cp etc/barnyard2.conf /etc/snort sudo mkdir /var/log/barnyard2
Next, we'll edit the barnyard2.conf file.
sudo vim /etc/snort/barnyard2.conf
Look for "#config hostname:  thor" and replace that with:
config hostname: localhost
Look for  "#config interface: eth0" and replace that with whatever interface you want to listen on:
config interface: eth1
Look for output database, and below that section add a line, BUT don't let it wrap!:
output database: alert, mysql, user=snort password=yourpassword dbname=snort host=localhost

Starting Snort and Finishing Barnyard Config

Ok, let's test this puppy, at this point we can open snort on any active interface. If you have a machine with a single network interface, you'll use eth0 typically.

sudo snort -c /etc/snort/snort.conf -i eth1
It will take a bit to start, but if you get to:
--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.8.5.2 (Build 121)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/snort/snort-team
Copyright (C) 1998-2009 Sourcefire, Inc., et al.
Using PCRE version: 7.8 2008-09-05

Rules Engine: SF_SNORT_DETECTION_ENGINE Version 1.11 <Build 17>
Rules Object: web-misc Version 1.0 <Build 1>
Rules Object: web-client Version 1.0 <Build 1>
Rules Object: sql Version 1.0 <Build 1>
Rules Object: smtp Version 1.0 <Build 1>
Rules Object: p2p Version 1.0 <Build 1>
Rules Object: nntp Version 1.0 <Build 1>
Rules Object: netbios Version 1.0 <Build 1>
Rules Object: multimedia Version 1.0 <Build 1>
Rules Object: misc Version 1.0 <Build 1>
Rules Object: imap Version 1.0 <Build 1>
Rules Object: exploit Version 1.0 <Build 1>
Rules Object: dos Version 1.0 <Build 1>
Rules Object: chat Version 1.0 <Build 1>
Rules Object: bad-traffic Version 1.0 <Build 1>
Preprocessor Object: SF_SSLPP Version 1.1 <Build 3>
Preprocessor Object: SF_SSH Version 1.1 <Build 2>
Preprocessor Object: SF_SMTP Version 1.1 <Build 8>
Preprocessor Object: SF_FTPTELNET Version 1.2 <Build 12>
Preprocessor Object: SF_DNS Version 1.1 <Build 3>
Preprocessor Object: SF_DCERPC2 Version 1.0 <Build 2>
Not Using PCAP_FRAMES
Then you're up and running. Open a second terminal window and run this:
ls -la /var/log/snort
Look for 10 digit suffix on snort.log. If there is more than one file, copy the latest one. Create a file called barnyard.waldo in the snort log dir:
sudo vim /var/log/snort/barnyard.waldo
Enter the following, then save and exit:
/var/log/snort
snort.log
<10 digit number from step 2 above>
0

Start barnyard:
sudo /usr/local/bin/barnyard2 -c /etc/snort/barnyard2.conf \
-G /etc/snort/gen-msg.map -S /etc/snort/sid-msg.map \
-d /var/log/snort -f snort.log -w /var/log/snort/barnyard.waldo
Now, we need to test this out. The easiest way I think is to use nmap. This will do it (use the network tho that you are using, and keep in mind that if you're on someone else's network, they'll likely not like you scanning it). If you don't know how to use nmap at least a little bit, go learn something about it right after you're done here, linux.com has a good beginners tutorial):
sudo nmap -PN -v -O 192.168.1.0/24
If it starts correctly, you'll see alerts flow by in the terminal window. Check the base web page at http://127.0.0.1/base and see if you're getting alerts there. If you are, we're done.
On to part II?
Posted by bil at 8:01 PM
Edited on: Tuesday, June 15, 2010 11:13 AM
Categories: Other Software, Work
Comment by John Wan - Wednesday 03rd March 2010 06:33:18 PM

Hi Bil,

Thanks for all the good work,I am new to Snort, I followed the instructions on this url: https: / / wwwx.cs.unc.edu/ ~hays/ archives/ work/ index.php

All went well until I reached the following stage:

"Now a quick test. Run the following and see if snort runs: sudo snort -c / etc/ snort/ snort.conf -i eth1"

After running the command line above, I got the following outcomes:
Running in IDS mode

--== Initializing Snort ==--Initializing Output Plugins!Initializing Preprocessors!Initializing Plug-ins!Parsing Rules file / etc/ snort/ snort.confVar 'HOME_NET' redefinedPortVar 'HTTP_PORTS' defined : [ 80 ]PortVar 'SHELLCODE_PORTS' defined : [ 0: 79 81: 65535 ]PortVar 'ORACLE_PORTS' defined : [ 1521 ]Frag3 global config: .. .. .. .. .. +++++++++++++++++++++++++++++++++++++++++++++++++++

Initializing rule chains.. .ERROR: Undefined variable name: (/ etc/ snort/ rules/ bad-traffic.rules: 27): EXTERNAL_NETFatal Error, Quitting..

Is there anyone know what's this all about?

Would you please let me know "how to" fix this "Fatal Error"? what should I do next?

Any information and help would be much appreciated.

Thanks in advance.

Regards John
Comment by bil - Wednesday 03rd March 2010 07:03:36 PM

Sure, I'll take a look tomorrow. But I _think_ the issue might be that in the snort.conf file the EXTERNAL_NET variable isn't set. So you could try setting that variable. Try:
var EXTERNAL_NET any
And this might help:
Informit article on snort rules
BTW, I tried to email you directly but got a bounce back. bil
Comment by latgarf - Friday 05th March 2010 08:40:04 AM

Bil, Thanks a lot for the awesome tutorial! : )
Comment by Andy - Wednesday 10th March 2010 12:04:07 PM

Don't know if this is allowed here, but this link is a great help to people with troubleshooting their installs. It's helped me tons.

It's a how-to guide on setting up Snort and BASE on Ubuntu

http: / / ubuntuforums.org/ showthread.php?t=919472

Comment by Rob - Friday 12th March 2010 05:38:26 AM

Thanks for the great tutorial, everything is working fine : )

Just curious about the 10 number digit, do I have to edit the barnyard.waldo everytime I restart snort?
Comment by bil - Friday 12th March 2010 05:45:56 AM

No, in fact if you look at the waldo file after you've started snort, it should now be a binary file--for barnyard2, the text file is used as a "seed" to get things started.
Comment by Rob - Wednesday 17th March 2010 05:52:28 AM

Thanks for the quick response.

I'm running snort now for a few days, but maybe I'm a bit impatient, I'm really looking forward to part 2.

I'm a bit stuck now on how I can work with the rules, how to enable rules, how to manage the system so I really could do something with the alerts it gives etc.

It is really hard to find good tutorials on the net on what to do after the snort installation.

Again thanks for the great article and hopefully we can see part2 very soon.

Comment by Todd - Wednesday 17th March 2010 08:11:53 AM

Bil,

By far the best install I have ever found on the web. My only wish is that everyone should take note to the details and not leave anything to question or figure out for yourself. GREAT JOB!!!

Now, if I can find someone who can help me with SMTP on Ubuntu 9.10 which sends mail to another domain by logging in to it. Been working on this one at home for week and no joy. Tried PostFix, Sendmail with mutt and having a lot of issues.

Thanks for this Bil.. .. Oh, BTW, If we stop and restart snort, then I take it we need to add the new snort ID to the barnyard file, right? If, so I think I'll work on a script to automate starting snort, creating the barnyard file and then starting barnyard, that is unless there already is one. Any thoughts?

Todd
Comment by bil - Wednesday 17th March 2010 08:48:03 AM

Todd,
Thanks for the compliment.

In answer to your question, no, you shouldn't have to reseed the waldo file--if you stop and start snort, everything should just continue to work, since barnyard just watching the snort log. So far with barnyard2, I haven't noticed any issues, but in prior testing with barnyard last year, I did notice that if I stopped snort, I need to stop barnyard and restart snort first, then barnyard.

In terms of starting and stopping snort, there already is an init.d script in place if you used a pkg installation. Try:
sudo /etc/init.d/snort start

Also, I have a simple upstart script for barnyard, but I haven't done enough testing yet to post it. That'll probably be up by monday tho.
Comment by SnortUser - Friday 19th March 2010 02:38:44 PM

Thanks for all of your help in this. This guide was great, followed it step by step and I am up and off to the races, a HUGE thanks.
Comment by Jimbo - Tuesday 23rd March 2010 07:12:21 PM

Hi, Great tutorial.. I am running into one issue though.. I don't have an / etc/ snort.conf file just / etc/ snort/ snort.debian.conf that looks no where close to normal snort.conf file.. Do you know where i went wrong?

thanks again!
Comment by super newbie - Thursday 25th March 2010 12:00:25 PM

after I do apt-get install snort-mysql, an error screen comes up:

root@ubuntu: ~# apt-get install snort-mysql
Reading package lists.. . Done
Building dependency tree
Reading state information.. . Done
The following extra packages will be installed:
libprelude2 oinkmaster snort-common snort-common-libraries
snort-rules-default
Suggested packages:
snort-doc
The following NEW packages will be installed:
libprelude2 oinkmaster snort-common snort-common-libraries snort-mysql
snort-rules-default
0 upgraded, 6 newly installed, 0 to remove and 119 not upgraded.
Need to get 0B/ 2,523kB of archives.
After this operation, 16.2MB of additional disk space will be used.
Do you want to continue [Y/ n]? y
Preconfiguring packages .. .
eth0: error fetching interface information: Device not found


when I was using wireshark previously, I noticed that eth1 was my default and eth0 wasn't available as an option. I was able to do the commands following the install to create the database (and received the correct "Bye" response), however, when I try to move into the directory / usr/ share/ doc/ snort-mysql, I get this message:
root@ubuntu: ~# cd / usr/ share/ doc/ snort-mysql
bash: cd: / usr/ share/ doc/ snort-mysql: No such file or directory


Also note: I'm using VMware, so I know this disables eth0 - is there a way to get snort to either use eth1 or change eth1 to eth0? I'm a super newbie fyi.

Thanks so much!
Comment by Lolailo - Wednesday 14th April 2010 12:51:43 PM


THanks that is a great help!



I get these errors

Warning: include_once(Mail.php) [function.include-once]: failed to open stream: No such file or directory in / var/ www/ base-1.4.4/ includes/ base_action.inc.php on line 29

Warning: include_once() [function.include]: Failed opening 'Mail.php' for inclusion (include_path='.: / usr/ share/ php: / usr/ share/ pear') in / var/ www/ base-1.4.4/ includes/ base_action.inc.php on line 29 - I

$apt-cache search pear

I installed php-mail-mime php-pear

Then realized I miss php-mail

Comment by Jemiro - Tuesday 20th April 2010 02:58:20 AM

Thanks so much, i need this tutorial, ill use MySQL
: )
Comment by maravicasa - Tuesday 20th April 2010 12:01:17 AM

thanks
Comment by shiva - Wednesday 21st April 2010 03:27:46 PM

Hi bil,

I am using Ubuntu in sun virtual box and I followed your tutorial but at the end of the tutorial I found the following error, can you please find the solution for that error.

Thanks regards
shiva
--== Initializing Barnyard2 ==--
Initializing Input Plugins!
Initializing Output Plugins!
Parsing config file "/ etc/ snort/ barnyard2.conf"
Log directory = / var/ log/ barnyard2

--== Initialization Complete ==--

______ -*> Barnyard2 <*-
/ ,,_ Version 2.1.7 (Build 225)
|o" )~| By the SecurixLive.com Team: http: / / www.securixlive.com/ about.php
+ '''' + (C) Copyright 2008-2009 SecurixLive.

Snort by Martin Roesch & The Snort Team: http: / / www.snort.org/ team.html
(C) Copyright 1998-2007 Sourcefire Inc., et al.

WARNING: Ignoring corrupt/ truncated waldofile '/ var/ log/ snort/ barnyard.waldo'
Opened spool file '/ var/ log/ snort/ snort.log.1271249772'
ERROR: Unknown record type read: 2148576734
Fatal Error, Quitting..
Comment by Petter - Monday 17th May 2010 12:54:18 AM

Thanks for the info.

Quick question, any idea why I would get a "ERROR: / etc/ snort/ snort.conf(193) => Invalid keyword 'compress_depth' for 'global' configuration. when I run snort -c / etc/ snort/ snort.conf -i eth1??

I'm running Ubuntu 9.10 and Snort 2.8.6, and snort was compiled with --enable-zlib, and I did install the latest zlib from source.

I have spent time googling this to no avail.. .

Thanks for any help I can get.

-Petter
Comment by ndepoh - Wednesday 02nd June 2010 09:46:05 AM

Thank you sir
installation et setting up snort was my homework
with your document it was so easy
I am new in snort. it was my first time
it work correctly and so glad

excuse my bad English but I speak French
Comment by bil - Wednesday 02nd June 2010 09:52:44 AM

Merci. Sans aucun doute ton anglais est vachement meilleur que mon fran├žais.
Comment by Grant - Tuesday 03rd August 2010 12:24:00 PM

Hi Bil,

Great tutorial, thank you very much for taking the time to create this tutorial!

Thanks again,

Grant!
Comment by adhry - Saturday 29th January 2011 01:13:08 AM

Hi Bil,

Great tutorial,
I wanted to ask,
why alerts do not appear on its base,
snort when it goes well?
Comment by bil - Saturday 29th January 2011 05:12:10 PM

Well, I think the first thing to do would be to check the various logs to see if you can see any errors. You might also log in directly to the SQL db and see if it's being populated.
Comment by adi - Wednesday 02nd March 2011 03:51:37 AM

hi bil, i wanted to ask
i get a problem
unified2 fatal error, quiting
how can I solve the problem?
Comment by bil - Wednesday 02nd March 2011 07:55:02 AM

Can't say without more details, can you give me an outline of what you're doing when you get the error? Also, OS version, version of barnyard, etc?
Comment by Josh - Thursday 23rd September 2010 12:23:01 PM

Awesome job, I have a 100% working snort server!
Comment by Fafa - Saturday 04th August 2012 08:54:16 AM

Thanks a lot !!!
Comment by shad - Wednesday 21st November 2012 09:48:41 AM

I'm getting an error like this, any ideas on what i can do ?

FATAL ERROR: Failed to initialize dynamic engine: SF_SNORT_DETECTION_ENGINE version 1.16.18

Thank you in advance.
Comment by bil - Wednesday 21st November 2012 12:45:09 PM

Dunno. Found:
http: / / marc.info/ ?l=snort-users&m=120109371628312

Path idea sounds good. The other thing to try is to disable that module.

Do your prototyping in a virtual machine, like VirtualBox, take snapshots as you go.
Comment by Balaji - Monday 26th November 2012 11:58:36 PM

Hi Bil,

Great tutorial for a newbie. Thanks.
Everything works great until i reach the last step.
My barnyard.waldo file is like below:

/ var/ log/ snort
snort.log
1353991140
0

but when i run barnyard2 i get the following error:

--== Initialization Complete ==--

______ -*> Barnyard2 <*-
/ ,,_ Version 2.1.7 (Build 225)
|o" )~| By the SecurixLive.com Team: http: / / www.securixlive.com/ about.php
+ '''' + (C) Copyright 2008-2009 SecurixLive.

Snort by Martin Roesch & The Snort Team: http: / / www.snort.org/ team.html
(C) Copyright 1998-2007 Sourcefire Inc., et al.

WARNING: Ignoring corrupt/ truncated waldofile '/ var/ log/ snort/ barnyard.waldo'
Opened spool file '/ var/ log/ snort/ / snort.log.1353967120'
Closing spool file '/ var/ log/ snort/ / snort.log.1353967120'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353967297'
Closing spool file '/ var/ log/ snort/ / snort.log.1353967297'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353968421'
Closing spool file '/ var/ log/ snort/ / snort.log.1353968421'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353968548'
Closing spool file '/ var/ log/ snort/ / snort.log.1353968548'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353980386'
Closing spool file '/ var/ log/ snort/ / snort.log.1353980386'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353981965'
Closing spool file '/ var/ log/ snort/ / snort.log.1353981965'. Read 0 records
Opened spool file '/ var/ log/ snort/ / snort.log.1353991140'
Waiting for new data

Is there a solution for this?
Thanks a lot!!
Comment by bil - Thursday 29th November 2012 11:06:03 AM

The waldo file seems to be a tricky thing. It's been a while, but I recall having to try setting it up multiple times before it took. Sorry I can't be of more help.

Monday, March 15, 2010

Software to support the TEMPer USB thermistor

The TEMPer USB thermistor is a small unit you can buy online. The short version is that this is a cheap solution to having a computer record temperatures, but there are some issues. First of all, the company that makes these has used a variety of internals--early versions were straight serial devices converted to usb. Sometime in the last year or two, they switched to unit that use the HID interface, so you may need to search around to find code that will work with your unit. Also the software that comes with it is really bad, and support is almost non-existant, and the language barrier for english speakers is pretty severe. But the good news is the net being what it is, there are a lot of folks working on these things, and now there are some pretty good options.

Some links for background:

The last link is the one that was most useful to me, especially the part down in the comments--Magnus Sulland wrote a perl module that leverages libusb to read the device (many thanks to him!).

This document outlines installing his code on Unbuntu 9.10 (Karmic). I also tried this on Redhat and OSX, but no joy--the perl used on those systems is different (5.8.x) and this requires 5.10. Also, I think that using libusb might be tricky on OSX, since Apple handles some things very differently. If anyone out there has good installation for a TEMPer unit for OSX or Redhat, drop me a line.

First thing to do is install libusb's headers:

sudo apt-get install libusb-dev

Then we'll install some perl modules:

sudo cpan -fi Bundle::CPAN

If this is the first time you've run cpan, it will walk you through a series of configuration questions. I just took all the defaults and choose some local repositories when prompted. I'm sure that there's some way to tell cpan to not ask questions and just use defaults, but I don't know how (love to learn how tho).

Next, install the MakeMaker util:

sudo cpan -fi ExtUtils::MakeMaker

You may see some warnings that there are dependencies that have to be satisfied, just say yes. They'll look something like this:

---- Unsatisfied dependencies detected during [S/SI/SISYPHUS/Inline-0.46.tar.gz] -----

Parse::RecDescent Shall I follow them and prepend them to the queue

of modules we are processing right now? [yes]

I just hit return to accept the yes.

Once that's all done run these commands:

sudo cpan -fi Inline::MakeMaker 
sudo cpan -fi Device::USB

Now, install Magnus's module for the TEMPer device:

sudo cpan -fi Device::USB::PCSensor::HidTEMPer

The source for this is up at: http://search.cpan.org/dist/Device-USB-PCSensor-HidTEMPer/

Now, you need a file that will call the perl module, mine is named temper_mon.pl
cd ..
wget -O temper_mon.pl http://www.cs.unc.edu/~hays/dev/bash/temper/temper_mon.pl
chmod a+x temper_mon.pl
The file contains this code from Magnus Sulland into the file:
#! /usr/bin/perl

use 5.010;
use strict;
use warnings;
use Carp;
use Device::USB;
use Device::USB::PCSensor::HidTEMPer::Device;
use Device::USB::PCSensor::HidTEMPer::NTC;
use Device::USB::PCSensor::HidTEMPer::TEMPer; 
use lib;
use Device::USB::PCSensor::HidTEMPer;

my $pcsensor  = Device::USB::PCSensor::HidTEMPer->new();
my @devices   = $pcsensor->list_devices();
  
foreach my $device ( @devices )
   {
   say $device->internal()->celsius();
   }

Now, try to run it:

sudo ./temper_mon.pl

At first mine didn't respond. On one machine it started working on the third try. Another required a reboot. But in general, so far so good.

Note: After doing some installs, I started running into problems getting a segmentation fault when running the temper_mon.pl script. I got some advice from Magnus, and it turns out that the upgrade from Device::USB version 0.31 to 0.32 or 0.32 breaks this. To get around this issue in the short term, I downloaded version 0.31, built and installed that from source. I've put a copy of device-usb0.31.tgz up in case you need it. To use this, run the following commands:

cd
wget -O device-usb0.31.tgz \
https://wwwx.cs.unc.edu/~hays/dev/bash/temper/device-usb0.31.tgz
tar -xvzf device-usb0.31.tgz 
cd Device-USB-0.31/
perl Makefile.PL
make
make test
sudo make install

There's also a README file in the package with some information.

I've also written a short shell script to use with this as a nagios plugin and also a php program to use as a web logging tool, see Service Monitoring with the TEMPer USB thermistor for details.

Posted by bil at 8:07 PM
Edited on: Sunday, April 04, 2010 12:24 PM
Categories: My Software, Other Software, Work
Comment by internet - Sunday 09th May 2010 06:49:34 PM

I ran into this problem as well..
WARNING: Ignoring corrupt/ truncated waldofile '/ var/ log/ snort/ barnyard.waldo'
Opened spool file '/ var/ log/ snort/ snort.log.1271249772'
ERROR: Unknown record type read: 2148576734
Fatal Error, Quitting..
The fix was to configure bardyard as indicated on this site:
http: / / www.securixlive.com/ barnyard2/ faq.php


I also had a problem where Base-1.4.4 only displayed the main screen, none of the sub pages showed up. After turning on debugging in the php.ini i found the error Fatal error: Call to undefined method ProtocolFieldCriteria: : ProtocolFieldCriteria() in / var/ www/ base-1.4.4/ includes/ base_state_citems.inc.php on line 1113 I traced the problem back to something that was resolved in the Base-1.4.5 release Using 1.4.5 resolved the problem. http: / / sourceforge.net/ projects/ secureideas/ files/
Comment by Aaron - Thursday 26th August 2010 11:33:10 PM

I was able to get the TEMPer to work on ubuntu 10.04, the first try, using your instructions. Thank you very much for the work you've put into this howto.
Comment by jnihil - Thursday 02nd September 2010 08:24:44 PM

Likewise, no issues. Thanks a lot for the great article.
Comment by Jack Myers - Monday 13th September 2010 10:22:29 PM

When I reach the step:
sudo cpan -fi Device: : USB
and it hits the tests, it maxes out 7 gb ram and 7gb swap, thrashes my drive badly, kills Firefox & any other programs running and fails just about all the tests. It eventually finishes but leave the OS in such a sad state that I have to reboot.
Not sure what is going on here.
I'm on Ubuntu 10.04 64 bit, AMD quad w/ 7gb ram. I see that others have been successful on 10.04 so I'm confused.

Any suggestions on how I can track down the problem? (I'm pretty new to PERL but have been programming in a couple of dozen languages for a living for the last 30 years. Generally, adding a new language comes easy to me though I'm just starting reading the O'Reilly Perl nutshell book).

Thanks for any suggestions you may have (and for the work you've done on this).

Jack
Comment by eclyptox - Wednesday 15th September 2010 06:36:32 PM

Hi! Yesterday I worked out to make it run as you say. But since today in the internal in celcius I only read 0.99609375.

I read the temperature several times, but the internal one only shows that number.

Thanks.
Comment by bil - Thursday 16th September 2010 01:46:17 PM

Sorry, these last two I can't be of much help with. You might want to check out this like:
http://relavak.wordpress.com/2009/10/17/temper-temperature-sensor-linux-driver/
There's an extended discussion there, and Magnus Sulland has updated the perl code, but I haven't tried that or gone to ubuntu 10. I'll post a note when I do about my results, should be doing that in a couple of weeks. Also, 0.99609375 I think represents a limit of some kind being exceeded.
bil
Comment by Baskin Tapkan - Saturday 05th February 2011 06:48:12 PM

Thanks for the great post. Worked out great. I am in the process of brushing up with Perl and write the results to a file using perhaps Cron. Any other automated ideas? : )

Cheers!

Baskin
Comment by Torsten - Monday 02nd May 2011 07:27:20 AM

Hello

i tried the TEMPer stick for a few days with good results.
But today i can only read out 0.99609375 as 2 comments above.
Some one known this problem ?

BR/ Torsten
Comment by bil - Monday 02nd May 2011 09:02:38 AM

Yes, I see similar behavior from time to time, usually just reseating the Temper USB device clears it.
Comment by zelotoh - Wednesday 01st June 2011 05:33:26 PM

Hi,

Do you know how can I execute temper_mon.pl without sudo rights?

Thanks in advance
Comment by bil - Wednesday 01st June 2011 07:34:08 PM

No, I don't, since it needs privs to access the temper device. But if you add your user id, or if you want to allow anyone on your server to use it without admin rights, you can edit the sudoers file with visudo to grant users the ability to run this or any other command without typing a password. If you check out the followup to this article, there's some discussion of how to allow a user npre to run plugins without a password, and an example. Feel free to email me if you have questions,
bil