Alarm device: program

Another option is to set a Script as an Alarm Device. As all scripts are being started as a SKOOR Engine server user, it may be necessary to adjust the sudoers file on the operating system to be able to execute them. Be VERY careful about the permission settings, to prevent system vulnerability. 

Parameter

Description

$1

Alarm ID

$2

Event Type

$3

State of the origin object

$4

Name of the origin object

stdin / $MESSAGE

Body of associated Alarm Template

Self-terminating command

To make sure that an alarm program will not accidentally hang the SKOOR Engine process, use the self-terminating string at the beginning of an external command:

(sleep 3; kill -9 -$$) &

Alarm device program - output to 3rd party interface

Via the alarm device type Program, it’s possible to output the alarm into a text file or database. The MESSAGE part can be configured with the alarm template (see section Add alarm template).

Alarm program for a CSV output

Short alarm device program definition:

OUT_FILE="/tmp/alarm_out_$1.txt"
MESSAGE=`cat -`
 
# ID=$1, Event_type=$2, Origin_state=$3, Origin_Object_Name=$4
echo "$1; $2; $3; $4; $MESSAGE " > $OUT_FILE

The  file generated on the operating system will have a content like the following after an alarm was raised:

91249; AlarmOpened; Major; All Services; 2017.08.25 10:51:25 (MET) AlarmOpened Major on SLO 'All Services' SLO 'All Services' is in state 'Major' as SLO 'Services Zurich' is in state 'Major' as Job 'Tcp' on 'SKOOR Server Winterthur' is in state 'Major'

Complex alarm device program definition:

OUT_FILE="/tmp/alarm_out_$1.csv"

# Header
echo "ID(0);EVENT_TYPE(1); TIMESTAMP_CURRENT(2); TIMESTAMP_OPENED(3); DURATION(4); COMMENT(5); CURRENTLY_ASSIGNED(6); SOURCE_OBJECT_TYPE(7); SOURCE_OBJECT_NAME(8); SOURCE_DEVICE_ADDRESS(9); SOURCE_DEVICE_NAME(10); SOURCE_JOB_INFO(11); SOURCE_OBJECT_STATE(12); SOURCE_VALUES(13); SOURCE_ALARM_LIMITS(14); SOURCE_S_ALARM_INFO (15); SOURCE_L_ALARM_INFO(16); SOURCE_F_ALARM_INFO(17); ORIGIN_OBJECT_NAME(18); ORIGIN_DEVICE_ADDRESS(19); ORIGIN_ DEVICE_NAME(20); ORIGIN _JOB_INFO(21); ORIGIN_OBJECT_TYPE(22); ORIGIN_OBJECT_STATE(23); ORIGIN_VALUES(24); ORIGIN_ ALARM_LIMITS(25); ORIGIN _S_ALARM_INFO(26); ORIGIN _L_ALARM_INFO(27); ORIGIN _F_ALARM_INFO(28); REASON(29)" > $OUT_FILE

# Data
echo "$1"`cat -` >> $OUT_FILE

An alarm template with the followind body definition was linked to the alarm device:

;%e; %t; %T; %d; %c; %q; %n; %a; %A; %i; %o; %s; %v; %h; %0; %1; %2; %-n; %-a; %-A; %-I; %-o; %-s; %-v; %-h; %-0; %-1; %-2; %r;

Resulting content in the file:

ID(0);EVENT_TYPE(1); TIMESTAMP_CURRENT(2); TIMESTAMP_OPENED(3); DURATION(4); COMMENT(5); CURRENTLY_ASSIGNED(6); SOURCE_OBJECT_TYPE(7); SOURCE_OBJECT_NAME(8); SOURCE_DEVICE_ADDRESS(9); SOURCE_DEVICE_NAME(10); SOURCE_JOB_INFO(11); SOURCE_OBJECT_STATE(12); SOURCE_VALUES(13); SOURCE_ALARM_LIMITS(14); SOURCE_S_ALARM_INFO (15); SOURCE_L_ALARM_INFO(16); SOURCE_F_ALARM_INFO(17); ORIGIN_OBJECT_NAME(18); ORIGIN_DEVICE_ADDRESS(19); ORIGIN_ DEVICE_NAME(20); ORIGIN _JOB_INFO(21); ORIGIN_OBJECT_TYPE(22); ORIGIN_OBJECT_STATE(23); ORIGIN_VALUES(24); ORIGIN_ALARM_LIMITS(25); ORIGIN _S_ALARM_INFO(26); ORIGIN _L_ALARM_INFO(27); ORIGIN _F_ALARM_INFO(28); REASON(29)
91261;AlarmSeverityUp; 2017.08.25 11:05:11 (MET); 2017.08.25 10:59:57 (MET); 5m14s; ; ; All Services; ; ; ; SLO; Major; ; ; 0; 1; 2; Icmp; localhost; SKOOR Server Winterthur; 877; Job; Major; ; Warning: if Any Error code <> 0 (No error) Major: if Any Packet loss = 100.00 %; 0; 1; 2; SLO 'All Services' is in state 'Major' as SLO 'Services Zurich' is in state 'Major' as Job 'Icmp' on 'SKOOR Server Winterthur' is in state 'Major';

Resulting CSV in Excel:

Alarm program for a DB output

DB definition for the following example:

/usr/local/bin/mysql -uroot -pxxxx databasename -e"create table alarmlog (ID integer(10),REASON varchar(20),STATE varchar(20),DEVICE varchar(60),MESSAGE varchar(180));"

Alarm device program definition (mysql):

MESSAGE=`cat - | tr "\n" " " | tr "'" "*"`
ID=$1
REASON=$2
STATE=$3
DEVICE=$4

(sleep 3; kill -9 -$$) &
echo $MESSAGE > /tmp/test_log.log
/usr/bin/mysql -uroot -pxxxxx databasename -e "insert into alarmlog (ID,REASON,STATE,DEVICE,MESSAGE) values ('$ID','$REASON','$STATE','$DEVICE','$MESSAGE');"

Example of a result in the mysql database:

MariaDB [databasename]> select * from alarmlog;
+-------+-----------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------+
| ID    | REASON          | STATE | DEVICE       | MESSAGE                                                                                                                             |
+-------+-----------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------+
| 91307 | AlarmOpened     | Minor | All Services | 2017.08.25 12:53:09 (MET) AlarmOpened Minor on SLO *All Services* SLO *All Services* is in state *Minor* as SLO *Services Zurich*...|
| 91307 | AlarmSeverityUp | Major | All Services | 2017.08.25 12:53:18 (MET) AlarmSeverityUp Major on SLO *All Services* SLO *All Services* is in state *Major* as SLO *Services Zur...|
| 91307 | AlarmClosed     | Ok    | All Services | 2017.08.25 12:53:24 (MET) AlarmClosed Ok on SLO *All Services* SLO *All Services* is in state *Ok* as SLO *Services Zurich* is in...|
+-------+-----------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------+

Alarm program for snmp-trap output

Alarm template body definition (no subject will be used for the trap):

EVENT_TYPE: %e; TIMESTAMP_CURRENT: %t; TIMESTAMP_OPENED: %T; DURATION: %d; COMMENT: %c; CURRENTLY_ASSIGNED: %q; SOURCE_OBJECT_NAME: %n; SOURCE_DEVICE_ADDRESS: %a; SOURCE_DEVICE_NAME: %A; SOURCE_JOB_INFO: %i; SOURCE_OBJECT_TYPE: %o; SOURCE_OBJECT_STATE: %s; SOURCE_VALUES: %v; SOURCE_ALARM_LIMITS: %h; SOURCE_S_ALARM_INFO: %0; SOURCE_L_ALARM_INFO: %1; SOURCE_F_ALARM_INFO:%2; ORIGIN_OBJECT_NAME: %-n; ORIGIN_DEVICE_ADDRESS: %-a; ORIGIN_ DEVICE_NAME: %-A; ORIGIN _JOB_INFO: %-I; ORIGIN_OBJECT_TYPE:: %-o; ORIGIN_OBJECT_STATE: %-s; ORIGIN_VALUES: %-v; ORIGIN_ ALARM_LIMITS: %-h; ORIGIN _S_ALARM_INFO: %-0; ORIGIN _L_ALARM_INFO: %-1; ORIGIN _F_ALARM_INFO:%-2; REASON: %r

Example of an alarm device definition:

MESSAGE=`cat -`
/usr/local/bin/snmptrap -v 1 -c public 10.1.1.131 . 10.1.1.134 6 0 "" . s "ID: $1, EVENT_TYPE: $2, STATE: $3, NAME: $4 " . s "$MESSAGE"

Resulting trap: