How do you choose the right test software? Choosing testing software is a difficult task. There are many packages available at varying prices, and it is not always easy to estimate what future requirements you will need to take into account. The choice of assessment software is one that you make for the long term. Therefore, consider the specific requirements that the package should meet now and in the future. Perhaps the number of candidates, users or assessment locations will increase as a result of internal changes. Criteria for selecting assessment software To help you consider this, we outline a number of points for your attention: Technology Security (GDPR) Roles and rights Modular construction Booking and payment Ease of use Types of questions Availability of statistical data Multimedia applications Integration with other software Purchase options 1. How is the technology set up? Will multiple people spread across different locations be using the test software? Think of test designers and question assessors. This requires a web-based system, where questions are stored in a central and well-secured location. In addition, it is important to ask questions about the security of the system. Is this properly organised? Especially when it comes to summative assessment is important. In the case of systems that are only suitable for formative assessment this security is not always optimal. Finally, it is important to investigate how the software behaves when a large number of candidates participate. Are the servers used suitable for this? 2. Is the software secure (GDPR)? To what extent is the system equipped to meet the requirements of the GDPR? Consider security, storage and the correct use of personal data. Do people have the option to view their personal data and can it be modified or deleted? Keep these data processing issues in mind when choosing assessment software. 3. How should roles and rights be organised? When setting up examination processes, there are usually different responsibilities and authorities. To what extent can you align with this using digital assessment software? You can ask a number of questions here: Who is ultimately responsible for the content of the examination database? Who is authorised to enter, check and revise queries? Who is authorised to make changes to the templates used and to make changes at the level of the entire test? Conduct your research into suitable testing software, determine your requirements and find out to what extent the system meets them. 4. Is the software modular in structure? One question you can ask yourself is: how is the testing software structured? Software can offer a huge range of options, but do you actually use all the functionalities in the package? When choosing software, consider how the package is structured and whether it is possible to use it in parts or modules. Consider, for example, a module for on-site testing, for supervision, for enrolment and registration, and a module for analysing data and distilling reports. Is there also a possibility to create a ‘light’-version, for example for formative assessment? 5. Is it possible to book and pay for an examination? How easy is it for candidates to book examinations, change registrations or make payments? Are there options for linking a website to the relevant examination software? And how can communication with candidates then be facilitated? 6. How user-friendly is the software? How easy is it to enter questions? Can this be done by relatively untrained people? It may then be very relevant to make the test system accessible to test constructors and question assessors and to give the different groups different rights for entry, revision and finalisation. In addition, it may be important to make it visible in the system which questions have already been finalised, which have not, and which have already been revised. How is the status of questions made visible? Another point of attention is to examine the system's capabilities for analysing the results. Does the system already contain formulas for calculating p and rit values, including for question types other than multiple-choice questions? Some systems require the user to do this themselves, while others have this option integrated as standard. Finally, as a user, you benefit greatly from a clear structure. How are the questions organised in the database? Can you see the structure immediately, or does it seem more like a puzzle? 7. Do you test using open and/or closed questions only? How can the subject matter best be tested, with open or closed questions? You set open questions if you are testing whether a student can formulate the answers themselves, or when it concerns a ‘higher order’ skill, such as analysis. If this applies, look for a system that offers this possibility. In addition, there is the question of the extent to which the method of correction allows for flexibility: is it possible to use multiple correctors? And are there various possibilities for arriving at the mark? 8. Are statistical data available? Not every assessment software system has the same capabilities for distilling statistical data. This includes p-, rit- and a-values, as well as data on scoring by markers and a representation of the probability of guessing per question. Sometimes you also need information about which candidates had a particular exam question in their exam. This applies, for example, if you use flexible testing and want to exclude one or more questions from the exam afterwards. 9. Is it possible to display multimedia? For different types of questions, it can be useful to use visual material in the form of illustrations and/or videos. How easy is it to use these in the software? What requirements apply in terms of image formats and file sizes? 10. Are links to other software possible? Is it likely that a link to other software will be required? This may be necessary if other programmes, such as Excel or another accounting programme, need to be used during the examination. Some systems offer this option. 11. What are the purchase options? In writing examination completely different possibilities apply than with digital testing. To what extent is it necessary to test independently of time and/or place? Consider the future-proofing of the system as well. What if candidates live all over the country or even abroad? Then it is wise to take this into account at the outset. When considering the assessment options, also look at the possibilities for dyslexics, visually impaired people, etc. Or at the possibility of taking an exam offline; there may be situations in which there is no internet connection. Which test software will you choose? Even though the choice is complex, there are several important aspects to consider that can help you make your decision. These are listed briefly below, but each system has its own detailed specifications. Ask the right questions and choose the testing software that suits you best. Sign up for our newsletter Find out more about assessment software and digital assessment in our newsletter.
