Getting Started

Quick Tips for Planning and Implementation

You, or your team, can get started with your project with the help of the steps presented here. Also consult the Introduction and Assessment Methods sections of this site.

If you would like some help from CEI in planning and implementing your teaching project, please contact ctchris@ust.hk.

Here are the simple procedures to help you start planning and implementing your AFL course:

Step 1: What are the outcomes to be assessed?

If you are working at the course level, then learning objectives and outcomes need to be aligned with those at the departmental programme level, which in turn need to be aligned with institutional objectives and outcomes for students. Objectives and outcomes can be seen as statements from two perspectives of pre- and post-course respectively. In education today, there is an increased emphasis on learning outcomes rather than learning objectives. Objectives and outcomes can be grouped into cognitive, affective and behavioural (or knowledge, attitudes and skills respectively). These are a number of taxonomies available for these that are hierarchically arranged in terms of their degree of complexity and the two most commonly used cognitive ones at the university level are Bloom's taxonomy and Biggs' SOLO taxonomy. Bloom's taxonomy actually covers affective and psychomotor domains, in addition to the cognitive domain. A list of verbs you can use in writing objectives for each of these three domains can be found at http://eduscapes.com/tap/topic69.htm.

Back to top

Step 2: What are the capabilities/skills (implicit or explicit) in the outcomes?

At university level, it is important to review the assessments to check to what extent, and where, they are meeting the higher order cognitive levels generally expected of any university student. We also quite naturally tend to focus on the cognitive dimension of learning outcomes as these are generally easier to measure and provide feedback on. However, it is important to remember the affective and behavioural domains too. For example, in our specified learning outcomes the notion of increasing the ability for learners to manage their own learning, which in turn implies the ability to critically reflect on, and learn from, previous learning, is often implicit. So if self-managed learning, critical thinking and reflective learning are things that we value in our students' growth, then how will we measure these? Also, how will we ensure that both learning activities and assessment methods involve these attributes?

Back to top

Step 3: Is the method of assessment chosen consonant with the outcomes and skills?

Probably the most common method of assessment is an exam containing a mix of multiple-choice questions (MCQs), short open-ended questions and essays. MCQs are often very useful for checking knowledge recall but to design MCQs that test higher-order thinking, such as analysis, synthesis or evaluation, takes a lot of effort and often considerable piloting. Consequently, we need to look at providing a variety of assessment methods both at the course level and at the undergraduate programme level. For a list of assessment methods see the Assessment Methods section.

Back to top

Step 4:Is the method relatively efficient in terms of student time and staff time?

There are obviously competing demands here, as the most valid and reliable types of assessment can take the most time. For example, the Hong Kong exams authority takes two years to develop an exam which involves initial test construction, peer review and revision followed by repeated cycles of review-revision, then piloting followed by evaluation (e.g. item analysis of MCQs to check for item discrimination) and final revision. Obviously, such a long process is not practical within a university but some practical alternative that retains sufficient validity and reliability needs to be developed. A balance needs to be struck between coverage of learning outcomes and practical constraints of time, resources and cost. If you have not viewed it already, see the section of the video given by Dr. Peter Knight in the Introduction section where he talks about this issue and provides some suggestions for efficient ways of assessing.

Back to top

Step 5: What alternatives are there? What are their advantages and disadvantages?

Today, it is possible to generate quite a large number of assessment alternatives when the wide range of assessment methods available to teachers are put together with the different sources of assessment (i.e. who does the assessing tutor, peer, self, other) and th different assessment instruments (i.e. types of criteria). See the section on Assessment Methods for this.

Back to top

Step 6: Does the specific assessment task match the outcomes?

At this stage, it is really more a question of which outcomes (knowledge, attributes and skills) are being measured in a specific task. How does this fit in with the other assessment for the course or programme? It is likely that not all outcomes will be assessed but we need to know which ones are not being assessed and have a justifiable rationale for not including them.

Back to top

Step 7: Are the marking schemes or criteria appropriate?

Are the actual marking schemes and criteria aligned with the learning outcomes? For example, Biggs' SOLO taxonomy, mentioned earlier, has five categories and this has been used as a means of letter grading assessments (A, B, C, D, F) such as essays and portfolios. As the SOLO taxonomy corresponds to the type of thinking that is displayed it can be quite easy to see the alignment of an assessment with the specified learning outcomes.

Back to top

Step 8: How will I evaluate the alignment of the assessment (during design and post delivery)?

This question relates to quality assurance approaches to the construction of assessment tasks. Following the steps above is a self-assessment quality assurance method that acn be used during the design phase of the assessment task. You could obviously bring in a colleague from within your discipline to give some feedback on the assessment. Also, colleagues from other courses could listen to your rationale for the assessment and the coverage of outcomes and provide feedback through this discussion. Perhaps for core courses, there may be a departmental programme committee that could be involved in this feedback. However, you also need to think about how you will evaluate the assessment once it has been implemented; reviewing how the questions were answered and again self, peer and student feedback could be solicited. You will also need to think about the action that will be taken in the light of this and how the results of the evaluation of the assessment will be disseminated (presumably as one element in the existing procedure for disseminating evaluation results of courses and programme design and delivery). Below are eight self-assessment questions for evaluating the quality of your assessment processes:

Back to top