and tips on how to avoid them.
The validation of examination is a topic that often comes up in quality assurance systems, quality marks and legislation. Sometimes it is also laid down who does this and what must be done during this validation. It therefore appears to be a formality. Why, apart from the rules, validate an examination? And can it not just be done more quickly and more cheaply? We provide a number of practical tips for validating an examination and discuss the pitfalls involved. Don’t fall for them and read on!
Validating an examination is the step between developing and conducting the examination. This involves assessing and registering whether an examination or examination question meets the quality requirements.
When a developer or development group has done their work, the examination is often already checked from a technical point of view. So what is the use of validation? Apart from the established rules, it is a message to the outside world that exams are checked before they are used. “There is always a control step” and “the examination is always seen by more than one pair of eyes” are frequently heard statements.
In addition to perception, validating an examination also has a number of formal characteristics:
Validating an examination is therefore an important part of the examination development process. But it also comes with its own specific snags.
We all know, of course, that a validator is not the same person as the developer, after all, you don’t mark your own homework. But what do you do in the absence of specialists? The higher and more specialised the level, the more difficult it is to find enthusiasts who can do this work well and who do not charge a top rate for it. An alternative is to have developers validate each other’s work, or better still: consider a higher budget or other financial motives. Perhaps validators would like to enter into pleasant cooperation, expand their network, develop on a personal level or gain a bond with your institute.
The temptation is strong to have doubts about the developer’s knowledge after part of the examination has been rejected. But is that justified?
Be aware that developing an examination requires a different thought process than validating it. The validator has the challenge and the ‘luxury’ of being able to look from a distance, while the developer is in the middle of the process and by definition can look less objectively. In addition, developers still have room to grow in their profession. Conclusion: Provide feedback on the findings and give recommendations for improvement. In this way we can all become a little better and we continue to develop ourselves.
Let’s be honest, who has never thought this after a long validation meeting?
Validating by email, however, often leads to the following problems:
It’s great when a validator comes up with immediate improvements to the questions submitted. Especially when the development is under time pressure. This considerably reduces the percentage of rejected questions. If the direction of the question changes too much, a new question may have to be developed without a separate validation. The advantage of viewing the questions objectively as validator then lapses. This makes mistakes more likely. So be alert.
Nothing has such major consequences as examinations that do not meet the examination requirements. If the level is too high or if it contains issues that do not meet the examination requirements, students and other interested parties will lose trust. They no longer consider the examination to be an honest or valid measurement. Focus not only on the technical and substantive details of the examination, but also on the examination as a whole. In the case of creating an item bank, which has been developed per item and not per examination: check the examination by means of a sample draw. Is the examination relevant and is it correct in terms of ‘look and feel’?
The importance given to validation in the creation of examinations is justified. In order not to fall into the ‘pitfalls’, we give you the following practical tips: