This is a bit of a follow on from the previous post about Risk Based Testing, entitled “Risky Business…”

As Peter K mentioned, risk based testing is a large topic and I would like to share my thoughts and a different angle on the topic.

I have been in the testing game for 25 years and let me tell you, Risk Based Testing is nothing new. It may have been formalised and there might be courses available but I can guarantee that most of you reading this post, practice it every day and have done so for quite some time, even if you may not know it.

Let’s break it down: “Testing” may be what you do for a living or something you are researching but it is the “RISK” aspect I am talking about. Most, if not all, things you do each and every day have some element of risk. Consciously or subconsciously you weigh up the pros and cons and make a decision based on probable advantages and disadvantages of many everyday tasks.

Let me provide a simple example that ties back via the RBT acronym. Here in Australia, like most other countries, it is illegal to drive while intoxicated with alcohol and we call the check they do a ‘Random Breath Test” (or RBT). You might get home without being caught by the police or having an accident but is it really worth it? What are you gaining by attempting it versus what is the potential risk and consequences. To me this is a ‘no brainer’ i.e. the choice is simple – don’t do it. Risk Based Testing is not so different.

Unless you have infinite time to test, then you need to make a call at some point on what you will test, how thoroughly you test it, and what you won’t test in the time you have available. Sure, there will be some bugs that will slip through. The idea here is to minimise the occurrences and impact to the users of your application/system when these defects do occur. How do you achieve this goal?

Up front, you should consider ‘when’ you should start doing any sort of risk assessment. Ideally this should start in the project planning phase. Unfortunately this rarely occurs and is out of your control as the Test Team is often formed much later in the project lifecycle.

Your second best option is during the requirements gathering. This is a great time for Testers to get involved (while others are building houses of cards, we can tear them down – but that’s a topic for another blog!) but again we often are called in after the fact. Despite this, you can improve what happens during this phase for the next project by teaching the basics of RBT and demonstrating the benefits.

Ok, you are eventually called in to form a Test Team and need a strategy for testing. Ideally there will be a Test Manager that will deal with this from a macro level, but you and your manager can utilise the same tactics for your allocated pieces of work. You need to look at two key aspects of what is being delivered – The business view and the technical view. Let’s look at the business view first, as this is typically the earlier phase of the project lifecycle.

The Business View

In most cases the business is there to make money and we need to understand what may prevent them from reaching that goal. Some functions will not actually make money but it might retain our customers and keep them from going off to our competitors. We need to look at the IMPACT of failure. This makes our job difficult as we cannot just refer to some numbers on a spreadsheet to assess the value of what we are trying to help them achieve. What makes our job even tougher, is trying to understand each stakeholders view of the ‘big picture’.

The only way to do this is via a group review. The more stakeholders, the tougher the task, but to make things easier each and every stakeholder needs to be given the same criteria so they can bat it out on level ground. Here are a few examples and they are very open to weighting (importance):

• Cost to stakeholder – Who’s paying for this project?
• What is the benefit of providing this service?
• What will it cost if we don’t provide this service?
• Do we need to provide all of this service?

Note that the words ‘service’ and ‘functionality’ are used in the same context here.

You may see a common pattern developing here that will be used when logging defects later in the project and any reference point developed here must be maintained i.e. priorities can change.

The Technical View

You now need to look at the second part of the equation, that being, how the requirements have been actually implemented. To do this well, you need to have established a good working relationship with the Developers. It would be great to have a single meeting with both the Business and the Developers but depending on the project size, this may prove impractical. This in itself may increase the overall risk to the project and may need to be escalated.

This review is not about the blame game; it is about helping the Test Team to focus their efforts where it is probably best focused (NOTE: If a project fails – WE ALL FAIL. Don’t play the blame game).

During this meeting you need to look at the PROBABILITY of failure.  Here are a few items that you might consider:

• Developer experience. Is the Developer new? How much experience does he\she have? Are we outsourcing to this company for the first time? How many times before has this been done?
• New code. This can require more testing to cover off exceptions as it is untried and unproven.
• Spaghetti code. This typically occurs for maintenance releases when someone previously inserted code to cover an exception. It hasn’t been altered but may trap something new as an error that was not considered when the code was originally created. The older the code, the more likely this could occur.
• New technology. Are we porting to a new database? A new operating system? Will your old code work still?

Conclusion

There is no magic formula that will solve the problem of “What do I test and how thoroughly do I test it?” that you are facing, but hopefully the above considerations will give you some guidance and help you achieve your goals.

Author – Peter Bolin, Senior Consultant.