Systems integration projects in IT management field typically do not follow software engineering methodologies and tools commonly used by software developers. There are number of reasons for this. Integration work is often after thought, integrators (like myself) may write code (mostly scripts) but often not software developers by training. The tools are also lacking since the integration often involves proprietary components and programming languages provided by the vendors. Forget about IDEs, refactoring, unit testing frameworks, etc. Still there are tremendous benefits in using whatever tools available.
We've been working on gathering sort of "best practices" with the objective to make them standard operating practices for ourselves. Looking into some of the techniques/tools/methodologies and learning from our software developers, some of our customers, and web 2.0 world (collaboration), we've identified a core set of practices that we think is not too ambitious: collaboration, version control tools, and automated acceptance tests
Collaboration:
We will use collaboration tools available on the web to communicate throughout the life cycle of the projects. Most of the projects we participate in involve multiple parties (vendors, other integrators, different customer entities, etc.) and traditional tools email and ms office just do not cut it. Google Docs or Zoho Office are ideal solutions to work collaboratively on documents. Forget about the generic MS vs Google, desktop vs web application debates. Web based office tools are great fit for our type of work where multiple parties from different entities need to cooperate.
The challenges? Lack of internet connectivity inside our customer networks (for outsiders) sometimes prevent us from accessing the web, making life difficult. Believe it or not, this problem is more acute in telecommunications companies that are in the business of providing access to the Internet. How is that for irony?
Another problem is legal/psychological barriers to using web based (not hosted) tools in the corporate world. Collaboration makes more sense if most parties participate and use the same tools.
In addition to the web based office tools we believe instant messaging tools are key enablers. Given the time sensitivity of most of the projects, sending an email and waiting for a response causes loss of productivity. IM between key participants can dramatically improve the productivity of everyone involved. I should also mention that one of our products RapidInformer is also an alternative here as it enables the team members to communicate with each other but puts the individuals in control of how (via email, IM, RSS, etc.) they want to communicate.
Version Control:
This may look obvious but needs to be said. You'd be hard pressed to find a software development team that does not use some type of version control system such as CVS, yet at least in my experience, in most systems implementation/integration projects a version control system is not used. The infrastructure is not there and people don't take the time to set it up.
We already do keep the files from customer projects in our internal CVS, but we often cannot use our internal system during the project since it needs to be shared with at least the customer, and likely others, and it may not be accessible from customer locations anyway.
Going forward, we will request access to a version control system from our customers during project planning phase. If it is not possible, we plan to use a CVS installed on a local vmware instance.
Automated Acceptance tests
Our software development team creates "acceptance tests" which test the product or module they develop. Acceptance tests are black box system tests and created from the "user stories" which define the requirements as defined by the customer.
The acceptance tests are automated and run frequently as part of the continuous integration process to ensure that we don't break something while we adding new features, and making changes etc.
Acceptance tests also serve as the embodiment of the requirements. They describe in code how the solution should behave, what the expected functionality is.
Automated tests are essential part of the software development methodologies, and they are just as essential for systems implementation/integration projects. It is not always easy/feasible to do automated testing due to proprietary languages/environments provided by the tool vendors in the IT management tools space such as IBM, EMC, etc. How do you write a unit test in Netcool probe language? Acceptance tests however treat the system as a black box as stated above, therefore it is easier to write automated acceptance tests as long as the system has some kind of interface (and not all tools do). For example, if you're developing code for a Netcool probe, you can write tests that send input to the probe and check whether the probe code has worked as expected or not by checking the event on the Netcool server. We've decided to collected helper utilities we've developed to do these type of tests in a framework so that we don't start from scratch and reinvent the wheel everytime. We use the groovy as the programming language, as it provides all the infrastructure to work with our products as well as Netcool and Smarts through the APIs.
In a future article, I'll go through how we write acceptance tests for a solution that includes Netcool object server and RapidInsight as the web based UI and enrichment solution.
