Archive | Uncategorised RSS feed for this section

Fancy footwork, feedback and fun!

9 Nov

I’ve been busy in October helping colleagues John Unsworth recording a brief vignette about devising learning and teaching strategies for a project he’s doing for the British Council in the Ukraine, and for Santanu Vasant (with Kay Sambell) on changing assessment and feedback post-Covid19 for a new blog series he and colleagues are preparing. You might like to see incidentally our article on the changing landscape of assessment, translated into Spanish (thanks Ceridwen Coulby of Liverpool University, who is going to use it for a British Council project in Peru). You can find it here: The-changing-landscape-I-June-Espanol.docx (42 downloads)

I’ve also been continuing my project with UCLAN on enhancing the student experience plus my work with Galway University supporting their academic promotions panel.

But even in my leisure time a pedagogic professor’s got to pedagogically profess! So here for your delectation is my latest post on what we can learn from the UK show ‘Strictly Come Dancing’ about assessment and feedback: I hope it doesn’t destroy your weekend nights!

What can Strictly tell us about assessment and feedback in universities?

I used to be a bit of a TV snob before lockdown, telling friends I rarely watched it because I was too busy (other than CBeebies with the grandchildren and the occasional police drama set in the North East, which is where I live). Covid-19 has dramatically changed my habits though, and to cope with the fear and uncertainty of the pandemic, Phil and I have decided to throw ourselves into watching the full season of Strictly Come Dancing. Decades ago I loved to ballroom dance myself, especially when expert uncles guided me round the floor, and now I adore the costumes, the glitter and the pizzazz, but most of all the ability to shut off for 90mins at a time from the gruesomeness of our current situation. What I hadn’t expected was how much my day job as an educational developer and pedagogical consultant with expertise in assessment and feedback is being informed by what I thought was purely a leisure pursuit!

This is an ongoing process (we are only part-way through the series) so I imagine I will have more to say later, and no doubt fellow experts and Strictly fans will have plenty to add! But here are my initial thoughts.

Who gets on to the show is a risky business, just like university admissions. Sometimes we take a risk on who to enrol, hoping they might improve from an unpromising start and the same is true of contestants (who would have thought Bill Bailey would do so well?). We are all too quick to judge what people can or can’t do, and so who is permitted to engage is often limited by the smallness of our minds.

The assessment judgment process can be grim for learners: watching the contestants waiting to hear if they will come back to dance the next week, we can see how badly they fear a negative judgment (xx I’m looking at you!) , much like our students for whom university degrees are likely to have much more long-term effects on their lives than a game show, in whose faces we can see the emotional impact of assessment (Clegg in Bryan and Clegg, 2019) writ large.

Criteria: the judges are giving scores based on a significant level of knowledge about technical aspects of dancing (‘connoisseurship’, Orr, 2010, ‘We try to merge our own experience with the objectivity of the criteria’) which is not available to non-specialists. They tend to use very different criteria about the dancers, such as being physically attractive or behaving in ways that are surprisingly beyond normal expectations rather than technical competence, which is why layperson’s judgment tends to differ so greatly from the judges and why Phil and I often yell at the judges when they get it wrong by our reckoning. What is also scary is how often we are able to jump to an impression mark, based on nothing more than gut feeling, and hold to it despite contrary expert views (remind you of anything?). And is there a halo effect which stems from expectations derived from previous performance (“but she did brilliantly last time, I can’t believe she got such low marks this time”!)? And depending on which judge delivers their verdict first, do other judges adjust their marks accordingly as certainly happens in university double marking when the first marker is much more eminent and experienced than the second (which is why each should mark independently and then compare scores subsequently).

Hidden agendas: a lot rests on the choreography the professional has chosen to showcase the contestant’s skills and what they choose can impact on how well the contestant performs: (not really much chachacha in your chachacha!). Costumes, hair and make up often impact highly on scores, which are all aspects of performance beyond the dancers’ control, which replicates some aspects of university assessment.

It’s also the case that some features of the competition are covert rather than explicit: we don’t know how much the public vote contributes to the decisions about who gets into the dance-off and this may make it seem unfair: in universities we argue that students are less likely to complain about fairness if the process is open and explicit (Sambell et al, 1997).

Peer assessment: the phone-in votes in some ways mirror some aspects of peer assessment: inexperienced peers sometimes tend to vote high to support friends and that is emulated in the way phone-in votes are sometimes cast on the basis of factors other than dancing skills. Over the series it’s probably we get better at making more criterion-based evaluations, but I guess sympathy for the struggling underdog and fellow feeling will continue to kick in (just like in HE assessment).

Feedback: Craig’s feedback is brutal (couched in what David Boud might call ‘final language’ giving learners nowhere to go) but informative, and Nicola was able to avoid leaving the show because overnight she learned from his formative feedback and upped her game in the Dance Off. Motsi’s feedback is less helpful: it’s largely encouraging but not that informative, but you can see that her positivity is encouraging for contestants getting disappointing scores from the others. Shirley’s can be seen as the Goldilocks evaluations: not predictably and melodramatically fierce as Craig’s, but equally based on rigorous standards, and with the warmth and empathy that Motsi shows. Arguably all university assessors should aim to be like Shirley?

What needs to happen if participants are to avoid devastating disappointment is that they need progressively to develop inner feedback (Nicol, 2020) whereby they are better able to understand what good quality performances comprise (Sadler, 2010), having learned (if they are wise!) from watching the judges’ critiques of their peers, and hence are better able to make evaluative judgments of their own dancing.

So sorry to readers of this blog if I have ruined the escapism of your Weekends by this intrusion of pedagogic perambulations: if you want to add your thoughts, why not tweet in response to my pinned tweet @ProfSallyBrown?

References

Bryan, C. and Clegg, K. eds., 2019. Innovative assessment in higher education: A handbook for academic practitioners. Routledge.

Nicol, D., 2020. The power of internal feedback: exploiting natural comparison processes. Assessment & Evaluation in Higher Education, pp.1-23.

Orr, S., 2010. We kind of try to merge our own experience with the objectivity of the criteria: The role of connoisseurship and tacit practice in undergraduate fine art assessment. Art, Design & Communication in Higher Education9(1), pp.5-19.

Sadler, D. R., 2010, Beyond feedback: Developing student capability in complex appraisal. Assessment & Evaluation in Higher Education, 35(5), 535-550.

Sambell, K., Brown, S. and McDowell, L., 1997. “But Is It Fair?”: An Exploratory Study of Student Perceptions of the Consequential Validity of Assessment. Studies in educational evaluation23(4), pp.349-71.

 

Long-term Leadership Legacies

1 Oct

I’m delighted that this publication is out now, including the chapter by me and Pauline Kneale. It’s nice to be able to contribute something on leadership and in particular in the case of our chapter leadership legacy. I hope that some people at least will find it useful.

Quality, Question resolution and QAA Scotland

24 Sep

On Thursday 17th of September I was invited to do a Keynote for an event for QAA Scotland on compassionate assessment, looking at ways in which post Covid we can answer some of the questions about making assessment better for both staff and students. This substantially drew on the work I’ve been doing over the summer with Kay Sambell. QAA Scotland have now provided a link to the recording which you can find here. (scroll down to the link to start the recording of the presentation)  https://www.enhancementthemes.ac.uk/evidence-for-enhancement/student-engagement-and-demographics/student-mental-wellbeing

I have to say QAA Scotland is one of the best organisations I’ve worked with for years in terms of making everything easy for me, being efficient and managing IT processes. Quality indeed!

Energising Enterprising Educators

9 Sep

Today  Kay Sambell and I presented a keynote at the International Enterprise and Entrepreneurship Conference online on the topic of Making assessment future fit: ensuring authentic assessment approaches in the light of Coronavirus changes to HE practice. Our key messages were that assessment must be authentic if it is to be fit-for-purpose, and that now is the best chance in our working lifetimes to change assessment for the better, having had to make rapid changes in the pandemic. The slides are here: Enterprise-conf-Sept-2020-SBKS.pptx (171 downloads)   We must never revert to the bad old ways again.

First steps for Freshers Friendly Facilitation

7 Sep

Plenty of universities are wondering what are the most urgent things to do to build a sense of community among new student starting this autumn in the event of another covert surge. My Wonkhe post sets out briefly my thoughts about the five most important steps to take In case campuses have to shut rapidly to make sure freshers can make a good start. The link is here buff.ly/2Rej4vH

Authentic Assessment Activities

25 Aug

Today it’s pouring with rain, almost biblical in proportions, but it’s been sunshine in the study where I’ve really enjoyed running a webinar for the Hong Kong Education University together with some colleagues from the Hong Kong Baptist University, on the topic of post-covid-19 assessment. The slides are here: Hong-Kong-keynote-Aug-2020-w.pptx (232 downloads)

It was really interesting to have questions about setting convincing and secure science and maths open book exams, and on assessing group work in remote conditions, and the most joyous element was a language teaching colleague, Dr Jain, who described having changed her assessment techniques to more authentic ones with the result that the students performed much better. She was worried this might be a problem but I congratulated her and said it wasn’t surprising that better assignments lead to better performance. I look forward to working with colleagues in Hong Kong again in the future, alongside my partner-in-assessment Kay Sambell.

 

Radical Rethinking and Re-visioning

21 Aug

Further to my 17 August posting, I am delighted to say that SEDA have now published with very rapid turnaround our paper written specially for them on “Changing assessment for good: a major opportunity for educational developers” where Kay Sambell and I argue that the Covid-19 situation provided educational developers with a once-in-a-lifetime opportunity to change assessment long term as a result of the changes that have had to be made in the short term because of the emergency.
We conclude that radical change of the kind we were told was impossible is achievable and desirable in these new conditions we now find ourselves in. We suggest that if we don’t build on these in the future, we betray not only the trust of our students but also the endeavours of hard-pressed staff who have worked around the clock to make assessments in this year happen for students who otherwise would not have progressed and graduated.We have to make assessment in this new era truly represent what research (and our hearts) tells us genuinely works!
SEDA-special-post-covid-assessment-aug15-w.docx (147 downloads)

Swanton’s Sketchnote of Sambell and Sally‘s Stuff

20 Aug

Kay Sambell and I are delighted to see that the brilliant Katrina Swanton of Edinburgh Napier University has done a sketchnote of our latest 22 August article (in my post below this, but link again here:  Writing-better-assignments-in-the-post-Covid19-era.docx (465 downloads) ). We admire her work so much that we asked if we could use it here!

Raspberries, (w)Riting, Reflection, Rage, Reading, Running around, Ruminating and Relaxing

17 Aug

Designing Better Assignments Post-Covid

Although I haven’t posted for 6 weeks, I’ve been pretty busy over the summer.

  • Participating in the formal defence of her thesis by my dear friend Dr Hetty Grunefeld at Utrecht University. I should have been there in person in the ancient university hall and celebrating with her and other members of the PhD by Publication Group I’ve been convening for years, but it wasn’t to be so I did it from my study in an approximation of academic dress as requested  and was overjoyed to see her graduate;
  • Running a three-day virtual writing residential for PebblePad clients on starting to get published on learning, teaching and assessment;
  • Leading a workshop for Leeds Beckett senior staff aiming to achieve Principal Fellowship of the Higher education Academy;
  • Mentoring half a dozen colleagues pro bono during lockdown to help them keep up momentum with their life and writing goals;
  • Drafting a chapter on doing a PhD by Publication for a volume expected to be published in 2021;
  • Completing and submitting my SEDA Senior Fellowship reflection, which is a condition of my ongoing status retention;
  • Continuing to work with Kay Sambell on our suite of publications on managing assessment during Covid-19 and beyond. We are in particular working on ensuring that HE assessment in the future never reverts to the ghastliness of largely-ubiquitous, traditional unseen time-constrained exams, instead using more fit-for-purpose and authentic assessment approaches. We hope that one of these will be published by SEDA in the Autumn and another will be posted on this site and Kay’s very soon.
  • We’ve also been working on a new post on ‘Writing better assignments in the post-Covid19 era: approaches to good task design’  Writing-better-assignments-in-the-post-Covid19-era.docx (465 downloads) which I hope colleagues will find useful when moving away from traditional exams. While we’ve been writing it , I’ve been driven to rage and fury at the current UK totally-foreseeable problems our government has inflicted on our nation’s A-level students by applying a flawed algorithm to predicted results for students applying to universities right now, resulting in devastating injustice, reinforcing my strong reservations concerning exams!

Meanwhile Phil and I have been coping with domestic lockdown with plenty of home-based productive activities including lots of fruit harvesting (fabulous raspberries, redcurrants and cucumbers this year) and produce making, reading novels (Ali Smith’s Autumn, Winter, Spring and Summer are what I’m reading at the moment), socially distanced interaction with our Newcastle grandchildren including garden yoga classes, much baking and socially-distanced outdoor catching up with friends.

In the Autumn I’m looking forward to more writing with Kay, continuing to work with NUI Galway on their academic promotions scheme, a couple of virtual keynotes (I’m getting better with Zoom but still struggling with Teams despite a broadband upgrade) and working with another university helping them enhance NSS scores, following successful outcomes from my collaborations at Solent University last year.

This is my seventieth year and I had planned to spend much of it traveling abroad, which obviously isn’t happening, but I’m still managing to have a busy, engaging and largely enjoyable time.

PhD event, Publishing , Principal Fellowships and Pootling

28 Jun

I’ve got plenty going on this week and next despite Covid-19.
On Friday I’m participating online in a viva for my friend and colleague Hetty Grunefeld at Utrecht University: technically I am an ‘opponent’ but not a cause of stress for her or me as this role largely means I can ask questions to help her showcase her research in her replies.
I’m devastated to be missing the live event because in the Netherlands these are highly formal but celebratory events but I will do my best to make it a marvellous  event for her. I think all the Dutch participants will be there in person in their majestic Great Hall and even though I am joining in remotely, I am expected to wear full academic dress including a white blouse which I’m having to borrow. I will improvise with various hoods, hats and NTF stole that I have in the house!
And next week I’m running a remote event for Leeds Beckett staff on HEA Principal Fellowship as part of my regular commitment to help them with this area of work, and also running an event for Pebble Pad in Writing for Publication about Learning and Teaching as a replacement for my keynote which was to have been at their annual conference (see slides here: GetPublishedPebblepad-w.pptx (183 downloads) )
Also, I’m working with Kay Sambell on more resources on assessment during and post Covid-19. The three we’ve done so far are co-located in a single place here https://sally-brown.net/kay-sambell-and-sally-brown-covid-19-assessment-collection/ as some colleagues said they were finding them hard to locate on my site, and we will be adding to these over time.
After that I expect to go back to pootling on peacefully for the next few weeks.