An assessment strategy for assessing an online module or course

Introduction

What is the relationship between learning outcomes, module content and assessment? Although assessment was always an essential component of higher education, today it has taken on a much sharper and rigorous role based on transparency, fairness and justice. Since the Bologna Process (EU 2000) educationalists and academics have teased out these issues and developed national policies. It is outside the scope of this short paper to discuss issues such as programme validation, programmatic reviews, self-evaluation reports for programmes etc. This first part of this paper will examine assessment strategies based on Irish policy and the debates in the published literature. The second half of the paper will focus on a proposed assessment strategy designed for students in an online environment.

Assessment strategies

In order to ensure that standards are being upheld in higher level institutions and that students have achieved the learning outcomes of a particular module or programme assessment strategies are used. In Ireland, assessment standards are governed by the Quality and Qualifications Ireland (QQI))(Government of Ireland 2012) in a document entitled Assessment and Standards (HETAC 2015). This document states that quality assurance in Europe are benchmarked by The Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG). It argues that one of the most important aspects of higher education is the assessment of students. Assessment measures not only what the student has achieved but also the effectiveness of programmes. Assessments are meant to be appropriate, authentic and have clear and published criteria for marking. They determine how well a learner has achieved the learning outcomes. The awards which learners achieve are based on criterion-referenced assessments of learning outcomes (e.g. knowledge, skill and competance) (QQI, 2015, p.6). Modules and programmes should be coordinated. Assessments need to be fit for purpose and should reflect the required effort of the learner. Credits should be linked to the achievement of minimum intended learning outcomes.

According to Elwood and Klenowski (2002) assessment holds the key to student learning . However, assessment is not a straightforward process and involves judgments (what grade to assign a student to), decisions (what use will the assessment be put to?) and also the impacts on students (motivation and ensuring that students learn core knowledge) (Newton 2007). Newton argues that the professional discourse of education needs to be more precise. He examines the origins of assessment, which date back to Bloom (1971). Bloom argued that historically the purpose of assessment was to determine which students should not continue in the educational system. He posited a new approach – the development of the individual. Bloom provided definitions of formative and summative assessment, which have largely remained unchanged; they focused on three characteristics: purpose (use to which the information will be put); timing (formative assessment may be conducted at the end of every lecture or every few weeks) and generalisation (assess narrow or broad abilities). Formative assessment focused on small chunks of information covered in a module and to which the teacher provided feedback in order to improve learning. Summative assessment referred to the assessment at the end of a module after the learning had occurred (Bloom, 1971, cited in Newton, 2007). Later writers (Sadler, 1989) teased out the meanings of formative and summative assessment and concluded that the most important use of formative assessment was the use to which it was put. Other writers (Black, 1998 and Harlen and James (1997) cited in Newton, 2007) suggested that the distinction between formative and summative was blurred. Given that a person’s life chances may depend on the outcomes of assessments, Newton (2007) argues that assessment is a very complex process and ‘no one size that fits all’, therefore policy makers and educationalists should ensure greater transparency and should ensure that the assessments they use are highly reliable.

In terms of the impact of assessment on students Brown and Hirschfield (2008) conducted research (n=3469) on secondary school students’ concepts of assessments. They used self-regulation theory to interpret their data and found that students who self-regulate and take assessment seriously achieve more in terms of educational outcomes. Those who blame the schools or ignore the assessments and lack confidence achieve less (Brown & Hirschfeld, 2008). Torrance (2007) conducted large-scale research that involved gathering data from a wide variety of post-secondary education and training settings (in vocational education sectors). Data were gathered from 237 learners, 95 assessors and 260 other respondents (he does not specify who they were). He concludes that assessment has moved through a number of stages from assessment of learning, through assessment for learning and assessment as learning (Torrance 2007). Students are becoming instrumental in their approach to learning in that they focus on the assessment and the criteria needed to get a good grade. Consequently assignments and portfolios can appear similar in terms of lay out, structure format and types of evidence included. Tutors appear to be adept at coaching students, providing them with valuable feedback and criteria for successfully completing assignments. Torrance (2007) argues that this system may be doing the students a disservice because it weakens student autonomy and may lessen the challenge of learning. How can we determine the validity of assessments? Hargreaves argues that for an assessment to be valid it must engender a high degree of the desired learning (p. 186) and must be consistent with other social values in the twenty-first century (Hargreaves 2007). Hargreaves argues that collaborative assessment is valid provided that it promotes valuable learning. However there have to be some essential conditions such as:

• Classrooms focus on learning;
• Students’ and teachers’ familiarity with collaborative learning situations;
• Their familiarity with purposes and processes of the assessment;
• The appropriateness of the assessment task;
• And the meaningfulness of the ‘feedback’ constructed (Hargreaves 2007, p. 188)

In a collaborative situation, students would provide their own feedback and also receive feedback from peers and other adults. The focus is not on the individual but on the knowledge-construction group. Students collaborate with each other to ensure that they all understand the material. This is also called ‘peer learning’ and ‘cooperative learning’ and relies on cooperation rather than competition. Hargreaves acknowledges that this can be difficult particularly in a situation where there is a hierarchy and where the student believes that s/he can only learn from the teacher. The stated benefits include that students are more active, there is improved retention of knowledge on the part of the learner and they explore it in a deeper way. They use information in a flexible way and apply it in different situations and are more likely to become lifelong learners and do better in formal examination (Watkins et al 2001). Collaborative learning also helps to develop higher-thinking skills, the capacity to reflect critically, oral presentation skills, social skills and empathy. Despite all the benefits of collaborative assessment teachers are reluctant to introduce it into the classroom because of old-fashioned conceptions and expectations of what is meant by teaching and learning that focus heavily on individualism and social hierarchy. In order to improve assessment strategies Elwood and Klenowski (2002) advocate the use of ‘Communities of Shared Practice’ and communities of understanding where everything is out in the open and visible and students become assessors of their own learning. They undertook an action research project that studied their own teaching of the subject of assessment with students in a Masters programme on a module entitled ‘Assessment Issues’. They believe that criteria and grade descriptors should be made explicit to students. Discussion groups should be established where exemplars are provided to students who should be asked to assess this exemplar so that they are clear about how essays are marked. A goup consensus could be reached on the grade, students should be encouraged to assess their own and their peers assignments. Relevant feedback should be given to students in a timely fashion with suggestions about how they can improve their grade. Elwood and Klenowski (2002) conclude that assessment can only become an important tool for assessment when teachers and students grow a community of practice where everything in the assessment practice is visible and transparent and understood by both parties.

Constructive Alignment

Today educators in higher-level education settings are expected to engage in the practice of constructive alignment (Biggs 2009). Students are actively constructing knowledge and his/her own learning through relevant activities. According to Biggs (2009 p. 1), the key is that all components in the teaching system – the curriculum and its intended outcomes, the teaching methods used, the assessment tasks – are aligned with each other. The teacher must create an appropriate learning environment that perfectly matches the learning outcomes and methods, in which the student cannot help but learn. Teachers need to focus not only on the content but also on the level of understanding achieved by the students. Biggs suggests that there are four stages:

• Define the intended learning outcome
• Choose teaching and learning activities that will address the learning outcome
• Assess the learner’s achievement to see how they match the intention
• Grade the assignment

He suggests that there are two levels of knowledge: declarative and functioning. Declarative knowledge is second hand knowledge that the student can provide by writing or talking about it. However, declarative knowledge must transmit into functioning knowledge, e.g. doctors making informed diagnoses based on knowledge; lawyers making sound judgements based on legal principles; teachers engaging in sound pedagogical practices based on the most current educational theory and policy. Biggs (2009) suggests that this knowledge transforms the learners they see the world differently.

Part 2 – Assessment of an Online Module

The second section of this paper focuses on the assessment of an online module on a Master’s Programme in Teaching and Learning (Level 9, 5 credits of a 90 credit programme). This module is delivered online and consists of blended learning. There are asynchronous lessons, e.g. I wrote the lesson, a knowledge officer checks the content of the lesson and suggests edits to make it sharper, the lesson is recorded in a recording studio, instructional designers create high quality Power point slides. The lesson has knowledge checks and quizzes built in. The learner listens to the lesson and undertakes a multiple-choice quiz (not graded) at the end of the lesson; the learner can pick a number of responses, and the correct answers are given. There are synchronous tutorials where the content of lessons is teased out, and readings provided for learners; implications for practice are discussed. In addition students can engage in online forums where issues are debated. New questions are put on the forums each week, these questions correspond to the lesson. The learners are mostly primary school teachers and a few secondary school teachers. They are professionals in their own right and some are principals of schools.

Description of Module and Assessment

The module which I teach is entitled Intercultural Education. Intercutural education is based on the theories emanating from Sociology of Education and national policies on inclusive education. It is built around policy and legal imperatives which prohibit discrimination on nine grounds: gender, age, religion, race, Traveller status, marital status, family status, sexual orientation and disability. My colleague and I authored this module and have tutored and assessed it for the duration of the Master’s programme. The module is of 10 week duration and is assessed both formatively and summatively. The formative assessment consisted of a module exercise which was a proposal of what they intended to do (e.g. design an intervention or lesson plan) which teaches children the importance of social inclusion in relation to all aspects of intercultural education. The summative section consisted of an essay which incorporates a small piece of action research; they conduct a review of relevant literature on the selected topic, implement the intervention (or lesson plan) and then evaluate the success of the intervention.

Discussion and Conclusion

In this paper I have demonstrated how the programme learning outcomes, module learning outcomes and assessments are constructively aligned for the module I teach and assess. The assessment is designed to assess student’s knowledge, skill and competance of intercultural education; it has both formative and summative assessments. The overall aim of the Master’s programme is make good teachers better, and the overall aim of the Intercultural Education module is for teachers to understand how cultural diversity impacts on student learning. The assessment measures to what extent the teacher can engage in an effective pedagogy which include the principlies of intercultural education. In designing these assessments I have ensured that they are appropriate, authentic and that they have clear and published criteria for marking which will determine how well a learner has achieved the learning outcomes. The module is coordinated with the programme. The assessments are fit for purpose and reflect the required effort of the learner. Rich detailed feedback is given to students and those who receive a low grade are provided with guidance on how to improve their grades. In future, I hope to take Elwood and Klenowski’s (2002) advice to create a community of learners by including more active engagement by learners in the assessment process by providing them with an assessment to discuss and correct and by providing them with an exemplar.

Bibliograpy

Biggs, J., 2009. Aligning teaching for constructive learning, The Higher Education Academy. Available at: http://www.heacademy.ac.uk/assets/documents/resources/database/id477_aligning_teaching_for_constructing_learning.pdf [Accessed December 15, 2012].

Brown, G. & Hirschfield, G., 2008. Students’ concepts of assessments: Links to outcomes. Assessment in Education, 2008(15), pp.3–17.

Elwood, J. & Klenowski, V., 2002. Creating communities of shared practice: the challenges of assessment use in learning and teaching. Assessment & Evaluation in Higher Education, 27(3), pp.243 –256.

EU, 2000. The Bologna Process – Towards the European Higher Education Area, Bologna: European Union. Available at: http://ec.europa.eu/education/higher-education/bologna_en.htm.

Government of Ireland, 2012. Quality and Qualifications Ireland, Available at: http://www.qqi.ie/Pages/default.aspx [Accessed December 15, 2012].

Hargreaves, E., 2007. The validity of collaborative assessment for learning. Assessment in Education, 14(2), pp.185–199.

HETAC, 2009. Assessment and Standards December 2009, Dublin: Higher Education and Training Awards Council.

Newton, P., 2007. Clarifying the purpose of educational assessment. Assessment in Education, 14(2), pp.149–170.

Torrance, H., 2007. Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education, 14(3), pp.281–294.