Tag Archives: open badges

How we could reform assessment and certification

[Originally published on stuckwithphysics.co.uk on 5th November 2015]

In my recent post ‘Why we need to reform assessment‘, I outlined a number of issues which give me concern over the assessment of SQA National 3-5, Higher and Advanced Higher courses, introduced as part of the delivery of Curriculum for Excellence.

Whilst there may be many teachers who would wish for a return to the simpler assessment arrangements of the Intermediate 1 & 2, Higher and Advanced Higher qualifications of the ‘Higher Still’ era, which have been replaced by the CfE courses, I feel that one of the major shortcomings of CfE is its failure in doing anything to fundamentally change the nature or the purpose of assessment.

The new system, as with all of its predecessors, places almost the entire value of the certified qualifications on the terminal summative assessment of the course – the exam. Though some courses have significant elements of coursework, and many include an extended project or research task, still the majority of what our students, schools and staff are judged upon happens in a narrow time frame of a few hours at the end of almost an entire year of study. The results of these high stakes assessments supersede those for the individual unit assessments completed during the courses studied, rendering them effectively worthless.

In addition to this, there is the issue of the complexity involved in marking and recording the results of the unit assessments (which I outlined in ‘Why we need to reform assessment‘) which makes it difficult for students to understand whether they have passed units or not and increases the administrative burden on teachers.

In order to overcome these issues I propose the following changes to the methods of assessment and certification.

Unit Assessment via online e-Assessment

The SQA currently makes use of its own system for online e-Assessment, SQA Solar, for a range of courses across Nationals, Higher, HNC etc. Centres and candidates have unique, secure logins ensuring security of the assessment, and the system allows assessments to be scheduled at a time when the student is ready to be assessed.

This system could be expanded to incorporate all unit assessments in all subjects at all levels, and could be set up so that students performance could be recorded against the many individual criteria necessary to achieve a pass in a given unit. Any reassessment required could automatically be tailored to the specific areas not achieved at the first attempt. Given a sufficiently large bank of assessment items, or a sufficiently adaptable format, allowing numerical data to change for calculation-based questions (as it does on Heriot-Watt University’s Scholar VLE), it might be possible for students to make multiple attempts at assessments until the required standard is reached.

As the system is fully automated, this would free up teachers’ time for teaching and supporting their students learning, rather than using it for the bureaucratic administration of data. It would also reduce the ‘data chase’ required to ensure that SQA data is kept up to date on school MIS systems for transfer to SQA systems.

‘Points’ allocation and certification for internally assessed components –

Most courses have individual unit assessments which must be passed by students in order for them to achieve a grade in the final examination. Although these unit passes are included on students’ certificates, there is no explicit value placed upon them in comparison to the exam grade achieved. By allocating all components of all courses a number of points at the relevant SCQF level, students could potentially build up points across a number of courses whilst being able to choose whether or not to sit the final examination. This would reduce the ‘high stakes’ nature of the final examination, and allow for students, departments and schools to be judged and compared over the full range of their performance.

Points allocation for units could be based on the ‘size’ of the units, whilst exam grades could be allocated points determined by the band of pass. In my own subject, Physics, for example –

N5 – points awarded at SCQF level 5

3 x units, each with 10 points = 30 points,

Exam grade bands – A1 = 30 points, A2 = 25 points, B3 = 20 points, etc

Higher – points awarded at SCQF level 6

2 x full unit, each with 10 points + 2 x half unit, each with 5 points = 30 points,

Exam grade bands as for N5

Revision and separate certification of assessed course ‘Added Value’ units and ‘Assignments’ – 

Many courses have an internally assessed ‘Added Value’ unit, which at N4 has to meet every one of a significant number of individual criteria. Teachers are allowed to provide feedback to students in order to modify their submissions so that these criteria can be met.

The equivalent component of most N5 courses is an externally assessed ‘Assignment’, a formal report which is completed ‘under close supervision’ after a period of research which may include practical experimental work. Though guidance is given to students from their teachers, no feedback may be given on the report produced which is sent to the SQA to be assessed. The final mark for the assignment, given out of 20, forms a small proportion of the final score and hence the final grade.

These arrangements make it much more demanding for an N4 student, who may find the task much more challenging than most N5 students. A poorly completed N4 AVU would not meet all of the critera, resulting in the student not meeting the requirements of the unit, and subsequently not receiving an overall award for the course. A poorly completed N5 Assignment carries no such penalty, and would simply give the student a lower final score – without denying the student an overall pass.

Revisions should be made to the assessment of AVU tasks to make them fairer on the students. Perhaps an AVU could be consider to have been passed if a significant proportion of the criteria for the task, say 10 out of 15, were met by the student.

In addition to the significant differences in the assessments of these equivalent tasks across SCQF levels, AVU and assignment tasks are often very similar in related subject areas. This results in significant duplication of effort and repeated assessment of skills across a number of a student’s subjects.

By assessing these tasks on a skills basis, rather than within subjects, a single AVU or assignment could be completed by a student studying more than one science, or social subject. Students could choose which subject or subjects their assignments could cover, potentially allowing more meaningful, challenging, inter-disciplinary work to be undertaken. Though this might make the assessment of students’ reports more complicated, it might offer an opportunity to make the assessment criteria more flexible, as they are for the Baccalaureate qualifications undertaken by some students in S6. If nothing else, a reduction in the number of these tasks would significantly reduce the workload on students and reduce the SQA who have found it increasingly difficult to recruit sufficient markers for these tasks since their introduction.

I recognise that these proposals would require significant change to our current systems of assessment and certification, and that the Scottish teaching profession has experienced unprecedented change throughout the development and delivery of Curriculum fro Excellence. I further accept that one of the main reasons for avoiding radical change in the exam system has been concern that parents, employers, colleges and universities, might not fully understand the significance of new qualifications. In reality, it could be argued that these groups don’t fully understand the significance of the current qualifications system, and haven’t done so for a long time, if they ever have at all.

On a superficial level, it is easily understood that a student with an ‘A’ grade in a qualification is in some way ‘better qualified’ than another with a ‘C’ grade in the same subject, and that a student with five Higher passes is ‘better qualified’ than another with three Highers and two National 5s. But unless one has recently studied a course, or taught it, there is little chance of understanding what knowledge and skills are really involved gaining such a qualification, let alone how that qualification compares with other subjects or other levels.

It is often argued that we need these qualifications to allow universities to choose between applicants for places on their undergraduate courses. Without wishing to belittle this assertion, it does bear comparison to the ‘Sorting Hat’ in the Harry Potter novels – e.g. ‘AAAAB’ at Higher being the minimum requirement for a Law degree (Slytherin?). Increasingly, however, universities apply their own assessment requirements (BMAT, UKCAT exams), conduct entrance interviews, or consider applicants on the broader indicators of their personal statements, reducing their reliance on the crude measurement of ‘ability’ given by exam results alone.

In many ways the awarding of badges by organisations such as the Boy’s Brigade or Scouts to indicate the achievements of their members is a much more understandable form of accreditation. Indeed many professional and vocational qualifications are already ‘badgified’ in this way using industry standards, against which ‘badges’ are referenced and accredited. Mozilla, the organisation behind the Firefox we browser, support such a system for teachers to award ‘Open Badges‘ to their students using ‘open standards’ – where the criteria for which the badge is awarded are embedded as meta-data and awarded digitally. These badges can be electronically attached to a student’s digital profile via their blog, Google or other online account, and shared with prospective employers, colleges and universities.

Some work has already been undertaken by the SQA to develop this approach to accreditation, outlined in this press release from 2013, with small scale projects being adopted by some FE colleges, including Borders College, for accrediting both the work of students and staff CPD.

Open badges may not solve all of the short comings of our current system, indeed other, better systems may be in use elsewhere, or currently under development. Such a system, if combined with students’ unique Glow account could potentially stay with them throughout their schooling and beyond, perhaps even following them beyond further and higher education and into employment. The development of such a ‘Scottish Learner’s Account’, integrating assessment, certification and the accreditation of skills could form the foundation of a truly radical approach to these issues upon which students at all stages could build throughout the ‘Lifelong Learning’ that lies at the heart of the Scottish Government’s ambitions for the future of education.

Easy Openbadging

Last weekend, I took part in the Pedagoo event #tmlovelibraries. It was a fantastic day, and I learned loads. At the pub session afterwards, there was a sort of TeachMeet Unplugged event, similar in feel to the TeachMeet 365 events or, as Fearghal testified, to the very early TeachMeets themselves. Fearghal had asked us all to come with something we were prepared to share; as I have been doing a bit of work with OpenBadges and have been very impressed with them, I decided that this was what I was going to talk about.

Then I hit the problem. 2 minutes is not a very long time, particularly to talk about something you have been working on for months and have found out so much about. So, to keep things short, I decided to create an OpenBadge for all the participants of tmlovelibraries and then give it to them as a present. By claiming it, they could find out a bit about Openbadges themselves.

This idea seemed to work well in the keeping things short arena, as well as the engaging the audience area – the word ‘gift’ seemed to be the important one in achieving this! As Fearghal commented on the night, my talk also had the effect of taking his carefully honed structure and blasting it into a million pieces as people went scurrying to the internet to find their badge. The badge is shown below, together with its claim code for anyone who was there. To claim it, navigate to the badg.us site and insert the claim code ‘kapyua’ into the “Claim award from code” box. This will prompt you to either sign in to your Mozilla Backpack if you already have one, or sign up with an email address to create one before awarding you the tmlovelibraries – Participant badge, which you can then display on your blog, Facebook profile or Twitter feed.

In the impromptu break that followed my talk, I was talking to a few different people, and realised that there was a real appetite for finding out more about using OpenBadges. Quite a few people had looked at the concept themselves, before deciding that the project was too technical for them to use effectively. This, of course, is exactly the same decision I came to myself when I first started looking into digital badges. I had been impressed with the ease of creating badges for recognising various achievements on Edmodo, but had hoped for some way to display them in fronter, our school’s virtual learning environment. When I had approached the extremely helpful people at Edmodo asking if this was possible, they said that whilst they were happy for the badges to be displayed elsewhere, but it would need to be purely a case of copying them as an image and uploading them elsewhere.

I felt sure that there had to be a more efficient way of doing this, and went off doing a bit of digital badge research. It soon became clear that OpenBadges were exactly what I was looking for, but despite the fact that there were plentiful resources available for those with an ability to code, there was nothing I could find that was very user-friendly for a class teacher.

Until I chanced across the ForAllBadges site that is. Straight from the off, ForAllBadges allowed me to create an OpenBadge simply by uploading an image to the site and filling in the information fields to attach to it. Perfect for what I wanted. But ForAllBadges had far more to offer than I had been looking for. It gave me a whole badge-management system, allowing me to upload classes and add staff, create and issue badges and – most crucially given the age of my pupils – a way to display the badges earned without needing a Mozilla Backpack (currently, a Mozilla Backpack is only available to learners over the age of 13).

I soon had a pilot badge system up and running and a fronter page created with links to the pupil’s individual Trophy Rooms; here their badges could be seen through viewing their ForAllBadges badge journal. After an email exchange with the amazing people at ForAllBadges, the ability for the student to add a reflective comment to their badge journal was quickly added. This setup now allowed for a badge to be created, issued, displayed and reflected upon as well as having the advantage of being part of the OpenBadge system allowing a great degree of portability for the badges once the pupil reaches the age of 13 (or Mozilla update their terms & conditions to allow under 13s to have a Backpack with permission from their parent/carer – a change that is on the cards very soon I believe).

This was perfect for what I was looking to use it for in school, but perhaps a bit too complicated to use in ‘open play’. I had been thinking that OpenBadges could be a great way to document CPD activities such as TeachMeets or MOOCs for example, but how could an event organiser award a badge to someone whose details they didn’t know? Would they have to do all the data-inputting themselves? This sounded like a prohibitive amount of work.

Fortunately, a site that David Muir had pointed me towards had the answer. Badg.us allows a user to create badges very simply, and in much the same way as ForAllBadges. However, the badg.us site interfaces drectly with the Mozilla Backpack and Persona sign-in service, making it a far more user-friendly solution when you will be issuing badges to people from outwith your organisation or whose details you are unaware of in advance. It also lightens the administrative burden of issuing badges, as the onus is on the claimant to provide their details. The site allows you to set up reusable codes (like the one above) for large-scale issuing, or one-use codes when you are looking to target your badge claimants more precisely (I used this to create “Presenter” and “Organiser” badges for tmlovelibraries, printed up claim codes for these and gave them to Fearghal to distribute).

In my opinion, these tools make the whole process of creating and awarding badges far more accessible to the typical classroom practitioner; teachers who, much like myself and Fearghal, would previously have found the process too technical can use these services to gain the benefits of OpenBadges without having to become coding wizards. Other tools have been developed that can do a similar job – for instance, WPBadger and WPBadgeDisplay allow you to utilise WordPress blogs to issue and display badges whilst OpenBadges.me provides a very useful badge designer for either online use or as a WordPress plugin . Recently, the ForAllBadges site has joined together with its sister site ForAllRubrics, and you can set things up so that once a rubric has been com pleted, an OpenBadge can be awarded automatically. After some late-night Twitter conversations between myself and the founder of ForAllSystems, ForAllRubrics also has built-in links to the CfE Experiences & Outcomes. A very handy teacher toolkit!

So, now it begins to get exciting. The badges are no longer a concept. Now that a teacher – or a student? –  can create and award these badges, what might they do with them? I have a number of ideas that I’ll be trying in my school, and I know Fearghal had an inclination to use them as part of a programme he delivers at his school (this provoked a very interesting side discussion with David Gilmour about extrinsic/intrinsic motivation). I know that other organisations (including the Scout Association and – believe it or not – the SQA) have been looking at introducing them too.

What would you do with OpenBadges?

(this post can also be found on Iain’s blog (The H-Blog) at http://h-blog.me.uk/archives/435)