At this point you will have identified a system you wish to improve, brought together a team, formulated an aim, created a family of measures and then generated a series of interventions you would like to test. You’ve spent some time thinking about the data you will need and how to collect it, now it is time to test your change. This next section details the method in which you will test these interventions.
The Need for Testing
You’ve generated a series of interventions that you hope will improve your system. What is stopping you from rolling it out across the hospital? The truth is that not all interventions will result in improvement, some will make the problem worse. It is important to first understand how the proposed changes affect the system before you move onto the implementation phase. This is done through testing!
Testing can be achieved by using small scale plan-do-study-act (PDSA) cycles. This is explained in more detail below. The PDSA cycle enables rapid tests of change that quickly grow your learning while minimising the risks of introducing change into a complex system. Importantly, you can use it to identify issues or flaws within a project before you have gone too far down the road.
Someone Else Tested It
If the proposed change has been tested and proved elsewhere, it is still important for you to test it within your own system. This is because conditions will be specific to your situation and different to those in which the change was first tested. By testing during the introduction phase, you will learn how to adapt the proposed change to your particular situation, gain an insight into the costs of implementation and understand the side effects of the change. Involving the stakeholders in a PDSA cycle will reduce resistance to introduction and increase confidence that the change will result in improvement.
The PDSA Cycle
Consider the PDSA cycle as an experiment. You begin with a hypothesis (plan), run the experiment (do), interpret the results (study) and then revise the hypothesis (act). You then repeat the cycle with your updated hypothesis. Each element is described below.
The Plan section addresses how the change should be implemented and describes how the data should be collected. Address the following elements within your plan:
- What is the change you will be testing? What do you hope it will achieve?
- How will the change be administered?
- What information should be collected? Why is it important to be collected?
- Who will collect the data? This should be the same people within the system who are testing the change. This allows them to observe the changes and make adjustments as needed.
- Where, when and how will the data be collected?
- Who will analyse the data?
The Do section consists of performing the test of change on a small scale while collecting the data specified within the Plan section. It is important to have observers involved in this stage. Their role is to take objective notes and document any problems or unexpected observations. They should be physically present, attentive and non-judgemental. Consider the following questions:
- Did the test run as planned? If not, why?
- What happened when the tester tried to accomplish the task?
- What appeared difficult or awkward during the test?
- What steps are skipped or altered during the test?
The Study section involves completing an analysis of the data that has been collected and comparing it to your prediction. Summarise your findings and reflect on them. See the sections on Run Charts and Other Charts. Address the following:
- What happened during the test?
- How does this compare to your prediction?
- Has your change led to improvement?
The Act section is used to refine your change based upon information gathered from the test and analysis. Apply your learning and modify your change (as needed) before repeating the PDSA cycle. You may choose to expand the test, adjust the change or abandon the change altogether.
Read more from NHS Improvement on the plan-do-study-act cyle.
When a test does not achieve the desired outcome, this should not be viewed as a failure. In fact, it is a success of sorts because you will have been able to learn something from the test. It may even be more useful than a successful test. When a test fails, you will be able to learn in one of three ways:
- The test was not conducted as planned (i.e. you may need a better support system in place for execution of the test).
- There was a problem with data collection (i.e. you don’t know if the change resulted in improvement).
- The test was conducted as planned; however, the data doesn’t show improvement (i.e. you should think of new changes to test).
Every iteration of the PDSA cycle generates useful information that can be used to refine your proposed change. Repeated cycles allow problems encountered to be worked out and different strategies tested. Think ahead by planning for multiple tests across a wide range of conditions with different groups of people. Not every change will yield the same results when administered in different situations or with different people. These tests can be sequentially linked to grow your understanding of the problem. View it as a ramp of change that slowly increases your knowledge and confidence. Once you are happy with your proposed changes in a variety of conditions, you can then think about implementing the change into one location.
This may sound rather daunting, but the important point is not to get caught up in it all! Starting small is better than never starting at all! Begin with just one small change with one patient and one provider. You can work your way on from there.