Phased implementation of digital testing

How do you ensure a seamless transition from written to digital testing? You can treat it as a project, you can completely change the work processes or you can just start ‘small’ and implement digital testing in phases. But how do you get started? Below, we discuss some ways to get started on a small scale. This limits the amount of work and costs to start digital testing.

Impact of digital testing on existing processes

Start with a limited question bank

An easy way to keep it ‘small’ is to start with a limited question bank. We make a distinction between at random testing and fixed version testing. Later, as a next step, you can then easily expand the question bank.

Question bank for at random testing

In the case of at random testing, the starting point may be that each examination must be able to be taken three times with different questions. Why three times? In that case a candidate can retake the examination twice with completely new questions each time. If case of a third retake, he will be asked questions that he has already had, but in a completely different order and ‘shuffled’. The advantage of this is that the recognizability of a third resit is low.

Question bank for fixed version testing

Starting point for fixed version testing can be that you start with four examinations, a higher number than for at random examinations. Why increase the norm of the number of examination questions? Because something may be wrong with one of the examinations, making it temporarily unusable. Moreover, it gives the candidate a unique examination in case of a third resit.

Start with a ‘limited’ content of the question bank

If at the start of digital testing, you are opposed to using all kinds of different (closed) question types, you can of course decide to use them at a later stage. Then start with only the question types that you also use for written tests, for example open questions or a combination of open questions and multiple choice questions.

Fixed version testing

In the case of fixed version testing, the move to digital testing is relatively small. That is because you do not yet need to have the structure of the database and you can leave the development process unchanged. Testing with fixed versions is actually comparable to working with written tests.

Fixed testing moment

If you want to make the transition from written to digital even smaller, you can opt for a fixed testing moment. This is especially convenient when it comes to providing results and analysing results. Procedures concerning the determination and provision of results can then remain unchanged. If you want to change the result as a result of the testing, this can be done in groups, just as with written tests. With flexible testing you can also adjust the results, but this often only happens after candidates have already received their results. Just to be clear: Testing at a fixed moment is not the same as testing with fixed versions. You can also take tests at a fixed time with an at random question bank.

Part of the examination session in writing

If you want the examination sessions to be even more similar to the written version, you can take (part of) the examination session in writing. This can be done by making a printout of the digital examinations. The big advantage is that you can use the same locations as for the written testing. After all, no computers or a stable online connection are needed. You can then benefit from the advantages of digital testing, without having to change the processes surrounding the examination session.

An additional advantage of digital testing is that digital processing of the answers and analysis of the questions can also take place after written testing.

You can also have the given answers read in digitally during written testing. That is very convenient!

Introduce digital testing in phases via pilots

During the transition to digital testing, it is possible that new questions will be developed or even a completely new exam will be developed. In any case, it may come across as new in terms of ‘look and feel’ for candidates. How do you ensure a smooth transition? We distinguish the following:

1. Pilot with fixed testing moments
2. Pilot with trial period (flexible testing)
3. Pilot prior to opening.

1. Pilot with fixed testing moments

What does such a pilot look like? There are several ways to do that. The first, simple way, is to start with digital testing at fixed moments. For example, you can organize 2 or 3 fixed testing moments. After each moment you perform a question analysis and adjust your results, if necessary. In this way, you immediately test the quality of the questions and limit any unrest among candidates and other stakeholders.

2. Pilot with trial period

Another form of pilot, to be applied in the case of flexible testing, is to set up a trial period. For example, you can start with three months, starting with a lower cut-off point. You also communicate this to candidates. After that, you evaluate the pass rates and possibly go to a higher cut-off point. You can build up this increase gradually, in steps. This requires the necessary tact and clarity to prevent unrest.

3. Pilot prior to opening

Suppose you want to know how the questions score prior to opening the bank, then you can invite candidates to take an examination. Clear communication is required. What is the status if they pass? Will examination fees be refunded if they fail? Etcetera.