Assessment Methods

	Methods
1	Multiple Choice Questions (MCQ) Advantages Often referred to as 'objective' tests (in terms of their marking), this method can sample a wide range of knowledge and memory quickly and has the potential for measuring understanding, analysis, problem solving skills and evaluative skills. There are a wide variety of formats from true/false to reason/assertion. MCqs are easy to mark and analyse results. They are useful in terms of their ease of administration with large numbers of students, especially where marking is to be done by assistants rather than the test-setter. Computer marking and web-administered tests for mCQs are well-established. They are also useful for self assessment and screening. MCQs have potentially high reliability, validity and manageability, and feedback to students is fast. Disadvantages They are typically used to assess knowledge only. There is a danger of testing only trivial knowledge. More complex MCQ formats are not recommended: they confuse students unnecessarily and they are time-consuming to design. MCQs are not recommended if what you are trying to assess is an area in which there may be legitimate dispute about the "correct" answer. More demanding MCQs require more time to set. Better ones are based on case studies or research papers. To save time, look for banks of items on the Internet or in US text books. A team of assessors, working to the same learning outcomes, can brainstorm and produce several questions in an afternoon. For further advice on developing good MCQs see this site.
2	Short answer questions Advantages A standard method using slightly less structured questions than those in MCQs. They are frequently used in examinations to award a few marks as a 'starter', which is then followed by a question which requires more writing. Although they have the potential for measuring analysis, application of knowledge, problem-solving and evaluative skills, more often than not their construction means they only test the lower cognitive levels. Marking to model answers is relatively fast compared with marking problems but not compared with MCQs. They are convenient to use when a number of assessors will mark the papers, and all alternatives can be considered. Marking for feedback (e.g. with formative assessment) can be relatively fast. Disadvantages Easier to design than complex MCQs but still relatively slow. They also only tend to apply to convergent forms of knowledge. If they are used too frequently, students get into a learning habit of revising to produce the 'right' answers.
3	Single Essay Examination Advantages These are relatively easy to set and can test a wide range of abilities including capacity to draw on a wide range of knowledge, to synthesise and to identify recurrent themes. Marking for grading is relatively fast providing the criteria are simple. Disadvantages Attention to criteria needed and marking for feedback is relatively slow.
4	Essays Advantages These are applicable to any topic that requires the ability to construct and sustain a written argument. There are several types of essays that test different styles of writing and types of thinking. They can measure understanding, synthesis and evaluation, providing you ask the right questions. They are relatively easy to set and mark for grading based on impressionistic marking is fast. However, it is essential to determine the marking scheme before you start marking and then share it with the students — or better still, get them involved in drawing up the marking criteria/scheme and/or suggesting additional personal criteria. It is also worth having a formal student peer-review system on drafts prior to submission. Disadvantages They are inappropriate if what is being assessed does not require the ability to construct and sustain a written argument. Marking for feedback can be time-consuming, so you need to keep the criteria simple. Variations between assessors can be high - and so can variations of the Assessor. Essays undertaken out of class are insecure in terms of being vulnerable to plagiarism. Also, setting an essay early in a course, with a completion date at the end of the semester, is likely to give rise to surface learning where students focus their reading and efforts solely on the set topic, to the exclusion of the rest of the course content.
5	Cases and open problems Advantages These can range from simple vignettes illustrating issues in the practice of a discipline, through to complex sets of documentation which may require analysis and research. Solutions can range from short answer questions to complex plans or proposals. They have potential for measuring application of knowledge, analysis, problem-solving, evaluative skills and principles applicable to real-world situations. They can reach all the way up Bloom's taxonomy to "synthesis" and "evaluation" levels. Short cases are relatively easy to design and mark. Marking for grading and feedback are about as fast as essay marking. They can provide useful information for formative purposes, including diagnosis of problems, because answering the questions or meeting the requirements is often a multistage process. Cases are generally best used in conjunction with several other methods. They often overlap and move into simulations and they can also form the basis of project briefs, and lend themselves to collaborative learning. Disadvantages The design and development of more complex cases and their marking schemes are more challenging, and thus they are time-sinks! They also need to be piloted and revised in the light of feedback, which makes them problematic in terms of their use for examinations. Cases also require a degree of sophistication on the part of students when used at any level beyond the "single-issue" case and thus students need structured practice (e.g. progression from less open cases to more open ones). It is best not to use cases when criteria for assessment of correct or successful answers are contestable or unclear. But it is often better for there to be no single 'correct' answer. A key difficulty with writing cases is gauging the level of detail. If too little detail, the case may well suggest its own answers. If too much detail or too complex, then students tend to get lost. Distracting information is legitimate, but should be introduced with care (as with MCQ distracters). Don't let yourself get carried away with the production of the case story/history.
6	Modified Essay Questions (MEQs) Advantages A sequence of questions based on a case study. After students have answered one question, further information and a question are given. The procedure continues, usually for about one hour. They are relatively easy to set and may be used in teaching or assessment for developmental or judgmental purposes. They can be computer or paper-based and can encourage reflection and analysis. They have potentially high reliability, validity and manageability.
7	Presentations Advantages These can test preparation, understanding, knowledge, capacity to structure, information and oral communication skills. Feedback can be from any mix of tutor, self and peers. Marking for grading based on simple criteria is fast and potentially reliable. Measures of ability to respond to questions and manage discussion could be included. Disadvantages It is important that this assessment is not used when the skills of live presentation of material are not relevant to the course outcomes. Assessment criteria need to be clear, transparent and known to students from the start. If you allow students to use more high-tech options such as presentation software, with notebooks and data projectors, meaningful content may well get replaced by flashy animation and other presentation software effects.
8	Projects, Group Projects and Dissertations Advantages These are good for all-round ability testing. There is the potential for sampling a wide range of practical, analytical and interpretative skills, as well as the wider application of knowledge, understanding and skills to real/simulated situations. They can provide a measure of project and time management. Group projects can provide a measure of teamwork skills and leadership. Motivation & teamwork can be high. Learning gains can be high particularly if reflective learning is part of the criteria. They test methods and processes as well as end results. Projects allow students to contextualise ideas or material by applying them to practical instances. Disadvantages Marking for grading can be time-consuming. Marking for feedback can be reduced through peer and self-assessment and presentations. The use of criteria reduces marker variability but variations of challenge of project or dissertation can affect reliability. You need to ensure that students have the facilities or access to information in order to undertake a realistic project. If no clear project management framework is prvided (e.g. in textbooks or by tutors) and made available, students' projects are often sloppy. Another danger is the production of superficial marking criteria which are not effectively tied in to the learning outcomes of the course.
9	Poster Sessions Advantages These test students' capacity to present findings and interpretations succinctly and attractively. There is feedback potential from tutors, peer and themselves. Marking for grading is fast. Disadvantages There is a danger of focusing unduly on presentation methods but this can be avoided by the use of simple criteria. The use of criteria also reduces variability.
10	Orals Advantages These test communication, understanding, capacity to think quickly under pressure and knowledge of procedures. It is particularly useful when there is a need to find out about learning which has not been directly observed. Hence, the questioning focus which comes as part of a debriefing process after practice has been observed. There is great feedback potential with orals and marking for grading can be fast. Disadvantages Some standardisation of interview procedure is needed to ensure reliability and validity. However, it is extremely time-consuming and for this reason it may be carried out by associates of the main assessor. This reinforces the need for an interview procedure with standard questions. It is also essential that the questioning is carried out in a spirit of genuine enquiry, rather than accusation and put-down.
11	Mini-practicals Advantages A series of mini-practicals undertaken under timed conditions. They have the potential to sample wide range of practical, analytical and interpretative skills. Some, if not all, of the marking can be done on the spot so it is fast. Feedback to students is also fast. Disadvantages The initial design is time-consuming. They are reliable but training of assessors is necessary.
12	Reports on Practicals Advantages These have the potential for measuring knowledge of experimental procedures, analysis and interpretation of results. They measure know how of practical skills but not the skills themselves. Marking for grading using impressions or simple structured forms is relatively fast. Marking for feedback with simple structured forms is faster than without them. To reduce student workload and the assessment load, it is recommended to have a different assessment focus for each experiment. Disadvantages Variations between markers, without structured forms, can be high. This method is often overused.
13	Direct Observation Advantages This is useful for immediate feedback, for developmental purposes and for estimating performance - providing a simple, structured system is used. Impressionistic observation can be useful if supported by constructive feedback. It can be used by a group of peers to provide feedback as well as assessment. Disadvantages The presence of the observer can change the performance so the method should be handled sensitively. Intensive, lengthy training is needed for high reliability if detailed checklists are used. Reliability, validity and manageability are fairly high when structured observation is used. One observation is usually not enough, but there is a trade-off here as observation is a very expensive method of assessment.
14	Questionnaires and report forms Advantages A general method including a wide variety of types. Structured questionnaires get the information you want but semi or open-ended questionnaires may give you the information that you need. A mixture of structured and open-ended questions is recommended. Criterion reference grading recommended for judgmental purposes. Broad criteria are more reliable and valid than highly detailed criteria. Disadvantages Detailed criteria tempt users to react negatively or disdainfully.
15	Objective Structured Clinical Examinations (OSCEs) Advantages Oringally from the medical field, these are now used in business, legal practice, management and science courses. They are particularly useful for quickly assessing practical and communication skills. They can be used in an induction phase to estimate key practical skills. Group OSCEs are useful for teaching, feedback and developmental purposes. OSCEs can be used towards the end of a course to provide feedback or to test performance against outcomes. Reliability, validity and manageability are potentially fairly high. Disadvantages Fairly hard to design and organise but easy to score and provide feedback. They are probably less labour intensive than other forms of marking but several assessors are required at one time. Initially, they are time-consuming to design - but worth the effort.
16	Problems Advantages This method has the potential for measuring application, analysis and problem solving strategies. Marking for grading of easy problems is fast but marking of complex problems can be slow. Variation between markers is fairly low when based on model answers or marking schemes but you neeed to allow for creative, valid solutions by bright students. Disadvantages Complex problems and their marking schemes can be difficult to design. Marking for feedback can be slow.
17	Simulated interviews Advantages These are useful for assessing oral communication skills and for developing ways of giving and receiving feedback on performance. Peer and self assessment can be used. Assessment by simple rating schedule or checklist is potentially reliable if assessors, including students, are trained. Disadvantages Video-recorded sessions take more time but are more useful for feedback and assessment. Oral feedback on performance needs to be sensitively delivered so as not to damage self-esteem too much.
18	Learning logs/ diaries Advantages Wide variety of formats ranging from an unstructured account of each day to a structured form based on tasks. Requires a high level of trust between assessors and students. May have high validity if structure matches learning outcomes. Disadvantages Some training in reflection recommended. Time-consuming for students. Measuring reliability is difficult.
19	Reflective Practice Assignments Advantages Measures capacity to analyse and evaluate experience in the light of theories and research evidence. Relatively easy to set. Feedback potential from peers, self and tutors. Marking for grading is about the same for essays. Use of criteria reduces variability. Disadvantages Marking for feedback can be slow.
20	Portfolios Advantages A portfolio consists of evidence assembled to show how the student can meet specified learning outcomes or assessment criteria. There are a wide variety of types from a collection of assignments to reflection upon critical incidents. The latter are probably the most useful for developmental purposes. They can be used as the basis for orals. They have a rich potential for developing reflective learning if students trained in these techniques. The will be high on validity if the structure matches course learning objectives / outcomes. You need to ensure that assessment criteria go beyond simply requiring that a student has done something, and require evidence of understanding underlying principles. This usually requires an accompanying commentary which should link together the evidence in the rest of the portfolio, and relate it explicitly to the required outcomes. It is useful here to use a cognitive taxonomy for learning outcomes as developed by Biggs or Bloom (see the introduction section on this). Disadvantages Require a high level of trust between assessors and students. Measuring reliability is difficult. Portfolios can rapidly become enormous if clear guidelines are not given which will end up with time-consumed in finding one's way around them. Students need to be clear that a portfolio is to contain a selection of items with accompanied by reflective statements justifying their inclusion. Learning portfolios may well include poor work as a jumping off point as to what was learnt from this and a follow up 'good' item. It often helps to have an individual tutorial so that the student can orient the assessor to the content.
21	Work-based Assessment Advantages Variety of methods possible including learning logs, portfolios, projects, structured reports from supervisors or mentors. Validity, as usual, is dependent upon clear learning outcomes. Disadvantages Important to provide supervisors and mentors training in the use of criteria. Work experiences can be variable so reliability can be low.
22	Computer-based assessment Advantages Much talked about but still mostly of the MCQ and short-answer type which can mark and analyse results quickly. More recently this area has expanded to include mathematical problems and simulations. Reliability is high. Disadvantages Optical Mark readers can be used - but allow for some students not marking the items clearly. Time consuming to set. Validity (match with outcomes) needs careful attention.

50236 reads