Grading questions (arrangement questions) in (digital) testing: what is it and how do you deploy it?

There are numerous types of test questions you can use to create your ideal test. An example is the ranking question (ordering question). What are the advantages and disadvantages of this question format and what are tips you can apply when using it within (digital) testing?

What are ranking questions?

Ranking questions are questions that require the candidate to rank a number of options in order of importance, relevance or correctness. This could include steps in a process, events over time, or a list of items in order of importance. You can use this to test a candidate’s ability to organize and categorize concepts in a logical way. This tests the candidate’s acquired knowledge and levels of understanding at a different level than traditional multiple choice questions, for example.

Example

An example of a ranking question is as follows:

Q: Put the steps in the correct order you go through when buying a house:

1. Selecting a buying agent
2. Viewing a house
3. Making an offer on a house
4. Arranging funding
5. Sign purchase agreement

Answer options

In the example above, the answer options have already been put in the correct order. The observant reader can see that this ranking question creates confusion. Indeed, steps 1 and 2 can also be reversed and are therefore subjective. Therefore, make sure that you have the right testing knowledge!

Points

For each correctly ranked item, for example, you can award one point. For a completely correct sequence, for example, you can award a bonus point. It is important to adapt the scoring to the specific test and the emphasis you want to place on certain steps. It is also possible to accept multiple correct sequences, but with different scores. It is always important to establish clear and unambiguous criteria and lay them down in the review guidelines.

Advantages

There are several advantages of a ranking question:

• Flexibility: Grading questions can be used for different types of content and levels of mastery. This makes it a flexible way to (re)recognize knowledge of candidates.
• Exploration of knowledge: Grading questions force candidates to organize and categorize their knowledge and understanding. Especially when learning through formative assessment, this can provide additional insight and understanding of a particular topic.
• Practical Applications: Grading questions can be used with different topics and levels. This makes it a practical way to assess candidate performance.
• Improving memory: Grading questions force candidates to organize information in a certain way, which helps them remember and understand it better. This makes it an excellent tool for formative assessment.

Cons

There are a number of disadvantages to using a ranking question in digital testing:

• Complexity: Creating a good Grading Question requires some thought in terms of test construction. It is difficult to formulate answer options that are sufficiently specific but still allow for realistic ranking.
• Subjectivity: Ranking questions are often more subjective than other question types. This is because answers may depend on personal opinions and interpretations.
• Limited applicability: Grading questions are not appropriate for every type of topic or situation. Therefore, use different question types.
• Time: It can be time consuming to create and grade ranking questions. Indeed, there are more considerations and analyses to evaluate the answers. So tune this carefully to what you want to test!

Tips for creating a ranking question

• Keep the number of options limited: Try to keep the number of options limited to no more than six, as this helps the reviewer form a clear and manageable picture of the various elements.
• Use logical ranking options: Make sure the options are clear and logical to the candidate and can be arranged clearly and unambiguously.
• Addsupporting images and/or descriptions: Images and/or descriptions can help better visualize and clarify rank options. Be careful not to give unintended hints about the answer!

Conclusion – Grading questions (ordering questions) in (digital) testing

The ranking question is a unique closed-ended question type that can take your (digital) test to the next level. The big advantage is that they test the candidate’s ability to make logical connections between answer options. As a result, you measure better what you want to measure. The disadvantages of this test question are also there, especially in terms of test construction, you have to come from good houses. Want more information about a testing platform that allows you to easily use this question? Then read more about Optimum Assessment Platform.