Scenario: Customer uses Netcool Object Server for event management and requires the events to be enriched with data from a CMDB using RapidInsight. The device or link name will be used as the key to query the CMDB to get the information such as maintenance status, SLA level, location, etc. and populate event fields.
Typical development process would be to gather the requirements from the customer about how the enrichment should work, document the requirements if one is diligent and through, and move on to writing the code to enrich the events. RapidInsight does provide the necessary infrastructure to retrieve the events from the Netcool Object Server, pass to a script as name-value pairs. In the script simple operations can be used to get the data from the RapidInsight repository or external datasources, so we'd be left with coding the business logic of how to handle different type of events, when to do what in the groovy script. Once we code our logic, we can run it against existing events or create events using one of the probes and check whether the code works as we expect.
This post is about an alternative approach. Taking a page from software development methodologies such as test driven design, we can develop automated tests that would test the functionality of the enrichment solution. Automated tests are great for several reasons:
- Automated tests are code which means we can execute them over and over easily. The ability to verify whether everything works fine quickly is a liberating while developing code. It means we can iteratively enhance the functionality without worrying about breaking something because if we do, tests will tell us right away.
- Acceptance tests can serve as the "contract" with the customer. Better than any static document, acceptance tests describe the desired behavior of the solution.
- Automated tests can also serve as the basis for how to monitor of the solution once deployed in production. The automated tests enable application level monitoring.
In this post, I'll go through how groovy scripts (groovy4netcool library) can be used to write automated tests for Netcool implementations, as an example for the use of automated tests. To be able to do this test I need to do at least the following:
- create events in Netcool server.
- define different events for each type of event that should be treated differently
- get the event from the server and compare with the expected field values to verify whether the enrichment adapter has worked as expected.
I'll write the automated scripts using groovy. The groovy4netcool library gives us the ability to work with the Netcool object server easily as well as access to java libraries to do pretty much whatever we may need. The library also includes a simple testing framework. I'll write tests as separate self documenting scripts and execute them in order using the controlling test script. Here is how one of the test scripts would look like:
// me to work with Netcool server without having to deal with JDBC or SQL.
def ncAdapter = new NetcoolAdapter()
// the identifier of the event that will be created is testEvent1
// this will find the Serial for the event, IF the event exists
def serial = ncAdapter.getSerial("testEvent1")
// if serial is not null, then event does not exist
if (serial != null) {
//to start clean, remove the event
ncAdapter.removeEvent(serial)
}
// Groovy map structure (name-value pairs) is used to set the
// field names and values of the event
def fields = [:]
fields.Severity = 4
fields.Identifier = "testEvent1"
fields.Node = "NYCRouter1"
fields.AlertGroup = "Network"
fields.AlertKey = "Router event"
fields.Summary = "Created as a test event"
fields.LastOccurrence = ncAdapter.now()
//call the createEvent method to create the event
def newEvent = ncAdapter.createEvent(fields)
// check whether the event is created
assert newEvent.Identifier == fields.Identifier
//give some time for the enrichment adapter to do its work
sleep(3000)
// enrichment is supposed to populate the fields below
// if the values are not correct, enrichment did not work
// assert command throws an exception if the expression is not correct
assert newEvent.SuppressEscl == 1
assert newEvent.Severity == 5
assert newEvent.Location == "Brooklyn, NY"
The test scripts are executed by a test runner. The test runner script executes each test script one by one. If a script throws an assert exception, test runner catches the exception and moves to the next script.
runTest(new testRouterEvent2())
runTest(new testlinkEvent1())
def runTest(testInstance){
try{
testInstance.run();
}
catch(AssertionError ae){
println "FAILED TEST " + testInstance.getClass();
println ae.getMessage();
}
}
This testing framework can be extended to other tasks. For example, when writing rules file for a probe, the functionality of the probe can be tested by sending an input and checking the outcome from the server. Groovy's ability to leverage java libraries and pragmatic syntax makes it ideal tool. For example SNMP4J library can be used to send SNMP traps to mttrapd probe and created events can be checked as explained above, etc.
This type of testing becomes even more valuable when more than one product is involved. For example, RapidInsight is our web based UI solution for Netcool. RapidInsight retrieves events from Netcool using a connector. Using the same approach above, we can send events to Netcool and then check whether the events are available from RapidInsight web interface using HTTP methods available in groovy, testing the entire chain.
