Sally Brown

In the time Kay Sambell and I have been producing resources for our Covid Collection on assessment during and after the pandemic, many people have mentioned the difficulties of using authentic assessment processes in Maths to bring to life the creative possibilities and excitement of the subject. Here today we are delighted to offer you a special account by NUI Galway’s President’s Award winner Dr Giuseppe Zurlo in which he outlines a fascinating approach to engaging Maths students now and for the future.  Authentic_Assessment_gz-w1.docx (1252 downloads)

In March, Kay Sambell and I provided in our Covid Collection of articles on how to undertake assessment better this year when we’ve all been dealing with the backwash of the pandemic. Here we post a second collection which we are terming, after the terminology of the Star Wars films, the Authentic assessment compendium: Episode Two – the saga continues!  This contains a plethora of further examples of good authentic assessment, fifteen which have followed our established format that has proved of value to lots of people globally plus six more which were sent to us in different formats, but we thought still merited sharing. Even during the editing of this collection, we have been receiving more suggestions so it is possible that, in due course, we may well put out a third collection, potentially entitled Authentic Assessment 3: the Innovative Approach that Strikes Back at Boring Traditional Assessment (or something snappier!) Do let us know if you are finding these useful. (Corrected versions of this download added from time to time)  Authentic-assessment-compendium-Epiosode-two-the-saga-continues-2-may-w2.docx (2623 downloads)

(Thanks to Jim Edwards:  jimedwardspaintings.com for allowing me to use this wonderful image of the Great North Run on the Tyne Bridge with the Red Arrows flying by in the presentations Kay Sambell and I do together).

In March, April and May, Kay Sambell and I are leading a whole series of workshops designed to support staff who have made the complex transition in the last year, once face-to-face on campus assessment was not possible. A whole range of approaches were adopted and adapted by academics globally not only to cope with the contingency but also to improve assessment long term by ensuring that assessment is fully part of learning. Through the articles and presentations in our Covid Collection, we have argued that we can’t ever go back to former ways of assessment and so we propose approaches that fully integrate assessment and feedback with learning, leading to improved outcomes for students. To support these workshops, we have collected together a compendium of brilliant lightly-fictionalized examples collected from National Teaching Fellows and others in the UK and internationally, of what such alternative assessments might look like in different subjects, based on the approach we describe in our paper on 17^th August in the Covid collection on this website. Thanks to everyone who has helped us with this. Its first iteration is here,  Authentic-assessment-examples-compendium-w2.docx (3665 downloads) but we anticipate this will be added to over time as we encounter more similar super examples from different disciplines. Indeed, successive versions of this download are being added as corrections and additions are received – hence the change in ‘number of downloads’ field. Please note that example 3.2 was kindly provided by Thomas Broderick of Munster University of Technology.

Today I really enjoyed doing a virtual workshop on assessment and feedback for nearly 100 colleagues at NCG, which is an HE in FE provider in Newcastle, Carlisle, Lewisham, Southwark, Skelmersdale and Kidderminster. It was lovely to do a session for colleagues, many of whom live relatively close to me here, and to engage in a productive conversation. The slides for the session are here: NCG-Masterclass-10-Feb21-w.pptx (491 downloads)

Friends and colleagues of Phil Race will be delighted to know he is making steady progress in his recovery from a hip replacement three and a half weeks ago and he is looking forward to extending his walking range beyond the house and garden, currently constrained by snow. But it’s an ill (and chill North East) wind that blows no good, so instead of a photo of Phil striding up the street, here is one of our grandson Lucas eating porridge and doing elementary science in the garden, weighing snow (and then using the snow as giant snowballs!).

Just over a week ago Phil had his hip replacement and I’m pleased by how much progress he is making! He is doing all his exercises  and walking further and further outside every day. In the meantime life rolls on and I am currently finishing a chapter on doing a PhD by publication using a home-based networking group. Phil is now back on posting duties on my website so here are the slides used for a session today on demonstrating evidence of excellence and innovation in teaching and learning. NUIG-LT-IMcLSB27Jan21.pptx (563 downloads)

It’s been a hectic month pulling together a number of projects but I’m pleased to say that today I concluded my autumn semester work with a contribution to the ALTC online conference on assessment.
On Monday Kay Sambell and I ran a webinar for SEDA as part of their winter conference offering on authentic and compassionate assessment.
Prior to that I have been busy working with Galway University on recognising teaching and research excellence, and with the University of Central Lancashire on enhancing the student experience.
So it’s time to wish all friends and colleagues a very happy Christmas! It’s been a weird old year with lots of learning, a number of frustrations, and many things to be grateful for.. Phil and I would like to finish the year with this table of suggestions for anyone thinking of going for promotion on the grounds of teaching and learning this year which focuses on the kinds of evidence you should be collecting and collating to make a strong case. A link to our table is here: Demonstrating-excellence-in-T-and-L-final.docx (946 downloads)
After a complex and challenging year for everyone, we hope 2021 will bring health, peace, Joy, scholarship, collegiality and buckets full of laughter and fun.

I’ve been busy in October helping colleagues John Unsworth recording a brief vignette about devising learning and teaching strategies for a project he’s doing for the British Council in the Ukraine, and for Santanu Vasant (with Kay Sambell) on changing assessment and feedback post-Covid19 for a new blog series he and colleagues are preparing. You might like to see incidentally our article on the changing landscape of assessment, translated into Spanish (thanks Ceridwen Coulby of Liverpool University, who is going to use it for a British Council project in Peru). You can find it here: The-changing-landscape-I-June-Espanol.docx (583 downloads)

I’ve also been continuing my project with UCLAN on enhancing the student experience plus my work with Galway University supporting their academic promotions panel.

But even in my leisure time a pedagogic professor’s got to pedagogically profess! So here for your delectation is my latest post on what we can learn from the UK show ‘Strictly Come Dancing’ about assessment and feedback: I hope it doesn’t destroy your weekend nights!

What can Strictly tell us about assessment and feedback in universities?

I used to be a bit of a TV snob before lockdown, telling friends I rarely watched it because I was too busy (other than CBeebies with the grandchildren and the occasional police drama set in the North East, which is where I live). Covid-19 has dramatically changed my habits though, and to cope with the fear and uncertainty of the pandemic, Phil and I have decided to throw ourselves into watching the full season of Strictly Come Dancing. Decades ago I loved to ballroom dance myself, especially when expert uncles guided me round the floor, and now I adore the costumes, the glitter and the pizzazz, but most of all the ability to shut off for 90mins at a time from the gruesomeness of our current situation. What I hadn’t expected was how much my day job as an educational developer and pedagogical consultant with expertise in assessment and feedback is being informed by what I thought was purely a leisure pursuit!

This is an ongoing process (we are only part-way through the series) so I imagine I will have more to say later, and no doubt fellow experts and Strictly fans will have plenty to add! But here are my initial thoughts.

Who gets on to the show is a risky business, just like university admissions. Sometimes we take a risk on who to enrol, hoping they might improve from an unpromising start and the same is true of contestants (who would have thought Bill Bailey would do so well?). We are all too quick to judge what people can or can’t do, and so who is permitted to engage is often limited by the smallness of our minds.

The assessment judgment process can be grim for learners: watching the contestants waiting to hear if they will come back to dance the next week, we can see how badly they fear a negative judgment (xx I’m looking at you!) , much like our students for whom university degrees are likely to have much more long-term effects on their lives than a game show, in whose faces we can see the emotional impact of assessment (Clegg in Bryan and Clegg, 2019) writ large.

Criteria: the judges are giving scores based on a significant level of knowledge about technical aspects of dancing (‘connoisseurship’, Orr, 2010, ‘We try to merge our own experience with the objectivity of the criteria’) which is not available to non-specialists. They tend to use very different criteria about the dancers, such as being physically attractive or behaving in ways that are surprisingly beyond normal expectations rather than technical competence, which is why layperson’s judgment tends to differ so greatly from the judges and why Phil and I often yell at the judges when they get it wrong by our reckoning. What is also scary is how often we are able to jump to an impression mark, based on nothing more than gut feeling, and hold to it despite contrary expert views (remind you of anything?). And is there a halo effect which stems from expectations derived from previous performance (“but she did brilliantly last time, I can’t believe she got such low marks this time”!)? And depending on which judge delivers their verdict first, do other judges adjust their marks accordingly as certainly happens in university double marking when the first marker is much more eminent and experienced than the second (which is why each should mark independently and then compare scores subsequently).

Hidden agendas: a lot rests on the choreography the professional has chosen to showcase the contestant’s skills and what they choose can impact on how well the contestant performs: (not really much chachacha in your chachacha!). Costumes, hair and make up often impact highly on scores, which are all aspects of performance beyond the dancers’ control, which replicates some aspects of university assessment.

It’s also the case that some features of the competition are covert rather than explicit: we don’t know how much the public vote contributes to the decisions about who gets into the dance-off and this may make it seem unfair: in universities we argue that students are less likely to complain about fairness if the process is open and explicit (Sambell et al, 1997).

Peer assessment: the phone-in votes in some ways mirror some aspects of peer assessment: inexperienced peers sometimes tend to vote high to support friends and that is emulated in the way phone-in votes are sometimes cast on the basis of factors other than dancing skills. Over the series it’s probably we get better at making more criterion-based evaluations, but I guess sympathy for the struggling underdog and fellow feeling will continue to kick in (just like in HE assessment).

Feedback: Craig’s feedback is brutal (couched in what David Boud might call ‘final language’ giving learners nowhere to go) but informative, and Nicola was able to avoid leaving the show because overnight she learned from his formative feedback and upped her game in the Dance Off. Motsi’s feedback is less helpful: it’s largely encouraging but not that informative, but you can see that her positivity is encouraging for contestants getting disappointing scores from the others. Shirley’s can be seen as the Goldilocks evaluations: not predictably and melodramatically fierce as Craig’s, but equally based on rigorous standards, and with the warmth and empathy that Motsi shows. Arguably all university assessors should aim to be like Shirley?

What needs to happen if participants are to avoid devastating disappointment is that they need progressively to develop inner feedback (Nicol, 2020) whereby they are better able to understand what good quality performances comprise (Sadler, 2010), having learned (if they are wise!) from watching the judges’ critiques of their peers, and hence are better able to make evaluative judgments of their own dancing.

So sorry to readers of this blog if I have ruined the escapism of your Weekends by this intrusion of pedagogic perambulations: if you want to add your thoughts, why not tweet in response to my pinned tweet @ProfSallyBrown?

References

Bryan, C. and Clegg, K. eds., 2019. Innovative assessment in higher education: A handbook for academic practitioners. Routledge.

Nicol, D., 2020. The power of internal feedback: exploiting natural comparison processes. Assessment & Evaluation in Higher Education, pp.1-23.

Orr, S., 2010. We kind of try to merge our own experience with the objectivity of the criteria: The role of connoisseurship and tacit practice in undergraduate fine art assessment. Art, Design & Communication in Higher Education, 9(1), pp.5-19.

Sadler, D. R., 2010, Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550.

Sambell, K., Brown, S. and McDowell, L., 1997. “But Is It Fair?”: An Exploratory Study of Student Perceptions of the Consequential Validity of Assessment. Studies in educational evaluation, 23(4), pp.349-71.

I’m delighted that this publication is out now, including the chapter by me and Pauline Kneale. It’s nice to be able to contribute something on leadership and in particular in the case of our chapter leadership legacy. I hope that some people at least will find it useful.

On Thursday 17th of September I was invited to do a Keynote for an event for QAA Scotland on compassionate assessment, looking at ways in which post Covid we can answer some of the questions about making assessment better for both staff and students. This substantially drew on the work I’ve been doing over the summer with Kay Sambell. QAA Scotland have now provided a link to the recording which you can find here. (scroll down to the link to start the recording of the presentation)  https://www.enhancementthemes.ac.uk/evidence-for-enhancement/student-engagement-and-demographics/student-mental-wellbeing

I have to say QAA Scotland is one of the best organisations I’ve worked with for years in terms of making everything easy for me, being efficient and managing IT processes. Quality indeed!

Today  Kay Sambell and I presented a keynote at the International Enterprise and Entrepreneurship Conference online on the topic of Making assessment future fit: ensuring authentic assessment approaches in the light of Coronavirus changes to HE practice. Our key messages were that assessment must be authentic if it is to be fit-for-purpose, and that now is the best chance in our working lifetimes to change assessment for the better, having had to make rapid changes in the pandemic. The slides are here: Enterprise-conf-Sept-2020-SBKS.pptx (1183 downloads)   We must never revert to the bad old ways again.