Notification Resolvers

As we have already seen you can use all the provided Interpolations to format the data you use in Output Channels. But what if the provided data is not sufficient to connect to, for example, your third-party monitoring systems? One of the most common "problems" is that other monitoring applications do use other numerical or string representations for the severity of an issue than Avantra does. Avantra maps the numerical severity values from -2 to 2 to the string values Disabled, Unknown, Ok, Warning, Critical.

In order to cope with this kind of things easily, Avantra provides the so-called Resolver. A Resolver can replace the values of some standard Interpolations or add new Interpolations for use in Output Channels and Filter expressions.

Avantra provides three kinds of (Custom) Resolvers:

Simple Resolver: A simple Resolver allows to replace the values of the check status directly with values different from Disabled, Unknown, Ok, Warning, Critical. In addition, it allows you to replace newline characters easily with a different value which is a convenient way to fit, for example, values of Interpolations having line breaks into a single log entry.
Resolver with Executable: In a Resolver with Executable, you can provide your own program or script that adds new variables.
Resolver with JavaScript: The Resolver with JavaScript allows executing a JavaScript code to define own variables.

Defining a Simple Resolver

Perform the following steps in order to define a simple Resolver:

Procedure: Defining a Simple Resolver

Select Configuration Notifications from the top-level menu. Select the Resolvers tab.
Select New Resolver….
Fill in a Name and optionally a Description.
Choose Simple Resolver and push the Next button.
If you want to replace newlines, fill in a value for Replace `Newline´.

The following keywords are available in addition to any other character: [blank] for a space, [tab] for a tabulator, and [newline] for a newline itself.
If you want to replace any of the standard check status strings, fill in values into the fields Replace `Disabled´, Replace `Unknown´, Replace `Ok´, Replace `Warning´, Replace `Critical´.
Push the Finish button.

Defining Resolver with Executables

We first describe how to create a Resolver with Executable. However, we urgently recommend working through Resolvers.

Procedure: Defining Resolver with Executables

Select Configuration Notifications from the top-level menu. Select the Resolvers tab.
Select New Resolver….
Fill in a Name and optionally a Description.
Choose Resolver with Executable and push the Next button.
Fill in the Command Line.

You can include any of the available Interpolations in this field. Simply click and select the appropriate one from the list.

You may fill in a Working Directory and an Execution Timeout.
For every new Interpolation the executable defines perform the following steps:
1. Push the + button.
2. Fill in the Interpolation name and select the appropriate data type.
3. Push the Add button.
4. Repeat the steps for all new Interpolations.
Push the Finish button.

In order to reference the newly defined Interpolations in a Filter List or an Output Channels, you need to add the object name resolver, i.e. if you have added the Interpolation myText, you have to use it as ${resolver.myText} in an advanced Filter List or an Output Channel.

You will still be able to reference a newly created Interpolation myText as %%myText%%. However, this syntax is deprecated and you are strongly discouraged to do so.

In the old syntax, it was possible to override existing Macros. If your executable returned

CHECK_STATUS_STRING = FATAL

this value replaced the original value of the Macro.

While this is still possible with Avantra 23, it is as well deprecated and you are strongly discouraged to do so.

If you want to use the same Interpolation in a Filter or an Output Channel for Notification Actions with and without a Resolver, or you want to return new Interpolations in some cases only, make use of Global Expressions:

Define a Global Expressions checkStatusString like:

<#if resolver.checkStatusString??>
${resolver.checkStatusString}
<#else>
${check.statusString}
</#if>

and use it as <@ checkStatusString />.

Example 1. Example for a Resolver with Executable

In production environments, things are probably more complicated. Assume you have a third-party monitoring system that uses a more fine-grained severity, a string severity FATAL if, for example, the important Checks AGENTALIVE and SystemAlive are Critical in Avantra.

In order to achieve this, you need a custom executable or a custom script that adds new Interpolations depending on the condition mentioned above. As an example, we use a script that writes the following output

checkStatusString = FATAL
text = Check Status increased by custom resolver

in case one of the Checks mentioned above is Critical, and

checkStatusString = [the original value]
text = Check Status as provided by {avantra}

otherwise.

The following perl script may be used to achieve the requested results if it is called with ${check.statusString} and ${check.name} as arguments:

#!/usr/bin/perl
#
# Usage: check_status_resolver.pl ${check.statusString} "${check.name}"
#
# This script is called with the numeric check status provided
# by {avantra} and the check name.
# If the check name represents one of our <important> checks, and
# the ${check.statusString} is CRITICAL in {avantra} we
# return a new status with value set to 'FATAL'
#
# We also provide a modification.Text that indicates if we have
# changed any value or if it comes from {avantra} originally

use strict;
use English;

use constant MESSAGE_UNMODIFIED =>
'Check Status as provided by {avantra}';
use constant MESSAGE_MODIFIED =>
'Check Status increased by custom resolver';

# These are considered the most important checks:
my %importantChecks =
map { $_ => 1 } qw(AGENTALIVE SystemAlive);

my ($outputCheckStatusString, $outputResolverModification);

# Get the command line arguments and verify
my ($inputCheckStatusString, $inputCheckName) = @ARGV[0,1];
unless (defined $inputCheckStatusString and defined $inputCheckName) {
    print STDERR "Insufficient arguments\n";
    exit 1;
}

if ($inputCheckStatusString eq 'CRITICAL' and $importantChecks{$inputCheckName} ) {
    $outputCheckStatusString = 'FATAL';
    $outputResolverModification = MESSAGE_MODIFIED;
} else {
    $outputCheckStatusString = $inputCheckStatusString;
    $outputResolverModification = MESSAGE_UNMODIFIED;
}

print "checkStatusString = $outputCheckStatusString\n";
print "text = $outputResolverModification\n";
exit 0;

We create a Write into a log file Output Channel, the Resolver mentioned above and a Notification Action sending RealTime Monitoring Checks while using those two objects:

Write to notifier_action.log Output Channel

Name	`Log to notifier_action.log`
Support Action Types	Checks
File Name	`notifier_action.log`
Line Format	`${notification.date}: Status [${resolver.checkStatusString}] for [${system.name} [${check.name}]]: ${check.result} [Note: ${resolver.text}]`

Increase Severity for important Checks Resolver

Name	`Increase Severity for important Checks`
Macros Name	`checkStatusString`
Type	`String`
Name	`text`
Type	`String`
Command Line	`perl check_status_resolver.pl ${check.statusString} "${resolver.text}"`
Working Directory	`/syslink/resolver`

Log WARNING and CRITICAL RTM Notification Action

Name	`Log WARNING and CRITICAL RTM`
Define specific 'Ad-Hoc' Custom Selection Check Selector Type	`RTM Checks`
Check Search Criteria	Select check status Critical and Warning
Service Hours	`7 x 24 hours`
Notification Status	`CRITICAL`, `WARNING`
Output Channel	`Write to notifier_action.log`
Custom Resolver	`Increase Severity for important Checks`

We will run the Test using the following values:

Monitored System:	`PIP`
Check:	`SystemAlive [RTM] `
Suppress Output Channel Delivery:	cleared
Check Status:	`CRITICAL`

The output will be like:

INFO Mode: RTM Check.
INFO Load last check result:
Database server is paragon
DBCONNECT check is OK for PIP
Central instance is paragon_PIP_00
ABAP central services instance is paragon_PIP_01
J2EE central services instance is paragon_PIP_02
RFCCONNECT check is OK for paragon_PIP_00
J2EECONNECT check is OK for paragon_PIP_00
ASCS_MSGSRV check is OK for paragon_PIP_01
SCS_MSGSRV check is OK for paragon_PIP_02
INFO set last check date to: 2017-10-06 07:06:57
INFO set last status change date to: 2017-10-03 11:28:55
INFO Set check status to [2] ([CRITICAL])
INFO Processing action 'Log WARNING and CRITICAL RTM'.
INFO ----------------------------------- MESSAGE  -----------------------------------
INFO Action 'Log WARNING and CRITICAL RTM' applied to 'SystemAlive'.

INFO Message is:
INFO - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
INFO Database server is paragon
DBCONNECT check is OK for PIP
Central instance is paragon_PIP_00
ABAP central services instance is paragon_PIP_01
J2EE central services instance is paragon_PIP_02
RFCCONNECT check is OK for paragon_PIP_00
J2EECONNECT check is OK for paragon_PIP_00
ASCS_MSGSRV check is OK for paragon_PIP_01
SCS_MSGSRV check is OK for paragon_PIP_02
INFO - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

INFO Resolver 'Increase Severity for important Checks'.
INFO Executing command line: '[perl, check_status_resolver.pl, CRITICAL, "Check Status as provided by {avantra}"]'.
INFO Adding custom variable [%%checkStatusString%%] with value [CRITICAL]
INFO Adding custom variable [%%text%%] with value [Check Status as provided by {avantra}]
INFO No filter list set.
INFO Initial -> Scheduled: Enqueue message
INFO ----------------------- Run Output Channel, Attempt # 1  -----------------------
INFO Resolving following variables:
INFO OC ['Write to notifier_action.log']: logged 480 characters to file [/syslink/master/log/notifier/notifier_action.log].
INFO Scheduled -> OkSent: OC 'Write to notifier_action.log': Successfully sent!

And the output in `notifier`action.log_:

2017-10-06 07:07:16+0200: Status [CRITICAL] for [PIP [SystemAlive]]: Database server is paragon
DBCONNECT check is OK for PIP
Central instance is paragon_PIP_00
ABAP central services instance is paragon_PIP_01
J2EE central services instance is paragon_PIP_02
RFCCONNECT check is OK for paragon_PIP_00
J2EECONNECT check is OK for paragon_PIP_00
ASCS_MSGSRV check is OK for paragon_PIP_01
SCS_MSGSRV check is OK for paragon_PIP_02 [Note: Check Status as provided by {avantra}]

Works as expected.

Please review any Filter List eventually included in a Notification Action that also includes a Resolver. When adding values, you will probably need to adapt your Filter Lists.

Defining Resolver with JavaScript

Perform the following steps in order to define a Resolver with JavaScript:

Procedure: Defining Resolver with JavaScript

Select Configuration Notifications from the top-level menu. Select the Resolvers tab.
Select New JavaScript Resolver….
Enter a name and a description for the new resolver and click Finish.

The new resolver of the JavaScript type is added to the table of resolvers.
Double-click the JavaScript resolver to see its properties. In the Code field, you can define the output variable for this resolver. To assign values, use the javascript code: resolver.[macro-name] = [macro-value];

Example 2. Example for a Resolver with JavaScript

resolver.SNAssignmentGroup = "Default";

if ('${system.type}' === 'Server') {
    resolver.SNAssignmentGroup = "Server Group";
    if ('${customer.name}' === 'VIP Customer') {
        resolver.SNAssignmentGroup = "Server Group VIP";
    }
}
else if ('${system.type}' === 'SAP System') {
    resolver.SNAssignmentGroup = "SAP Basis Group";
    let checkName = '${check.name}';
    if (checkName.startsWith("PI_")) {
        resolver.SNAssignmentGroup = "SAP PI Group";
    }
}

Converting an Action into a confirming Action

There is one final feature of Avantra Notifications we have not yet covered. Notification Actions can be used to confirm RealTime Monitoring and Daily Checks.

It is not possible to confirm a single Daily Check detail, a Change or a CCMS Alert.

There are a few important things to note about confirming Notification Actions:

Only unconfirmed Checks can be confirmed. If a Notification Action is supposed to confirm a Check that is already confirmed, it does neither update the confirmation message, nor any confirmation time limit.
RealTime Monitoring Checks are only confirmed if they have not changed their check status until the time of confirmation. In theory, a Notification may be triggered and remain in a pending state for a while until it succeeds to confirm. In the meanwhile, the check status might have changed. In this case, the confirmation will be suppressed.
Notification Actions are processed by priority, i.e. they are processed in the order they appear in the Actions tab. If one of the Notification Actions has confirmed a Check other Notification Actions processed later will not confirm (this is virtually the first item rephrased).
Avantra confirms Checks with the built-in user messenger. There is no option to change this.
Notification Actions do only confirm if a Notification passed the Output Channel. If a filter blocks, the Notification Action does not perform confirmation.

Do not let the Notification Action confirm if the Notification Action itself is triggered by a Check Confirmation!

In order to make an existing Notification Action a confirming Notification Action, proceed as follows:

Select the Notification Action in question and open it. Unless you have defined a Notification Action operating only on Daily Check details there is a Confirm RTM/Daily Check: radio button.
Select Confirm on successful output channel message in case you want the Notification Action to confirm if one of the defined Output Channel succeeds
You may define a Confirmation Timeout. If so, the Check will automatically be unconfirmed after the timeout has passed. or if the check status changes.

If you leave the field blank the Check is confirmed until the check status changes.

This works for RealTime Monitoring Checks only.
Push the Save button in order to finish.