Is student course evaluation actually useful?

几乎所有当代大学课程结束后都要求学生给予反馈。但是,学生的反馈对改进将来的课程设计,甚至对评价教师是否有用?七位学者讨论了他们的经验

四月 16, 2020
Anatomy
Source: Getty

点击阅读英文原文


“学生们对我试图采取的教学方法和内容作何反应?我一直对此感到好奇”

在电子邮件发明之前的“远古时代”,当学生评价表格被发明出来的时候,我对此持怀疑态度。没有多少学生会费心去拿取并填写实体表格;而填写表格的学生常借此机会讲一些非常奇怪的事情。我的几个同事被求欢,而我的一位朋友——意大利最杰出的医务人员之一,也被反馈表激怒:学生说,他应该修剪头发了,因为现在的发型让他看起来像个嬉皮士。

但随着时间的流逝,我逐渐发现学生反馈也是非常有用的;前提是各方都认真对待它,并且管理者不将此作为生硬的工具。后者尤其危险,因为教学和研究都是晋升的指标。

当评论凸显出当代学生群体出乎意料的需求时,学生反馈尤其有用。学者们很容易忽视代际差异——这不仅是关于生活经历的问题(现在的学生把越南战争视为古老的历史,而我对参与反越战学生运动的记忆仍栩栩如生),而且还涉及了学生此前的学习经历。

例如,有一些学生在反馈中称在阅读模糊文本方面需要更多帮助。作为回应,我最近专门对此举办了一个特别研讨会。仔细阅读(分段阅读)是我们这一代人在学校中学到的,但并不是当今所有文学专业的学生都具备这项技能。如今的学生群体相当国际化;此外,学生和教师在信息、技能和兴趣方面的差异也确实很大。

Letter
Source: 
Getty

当然,反馈表上总有一些小偏差,并且没有证据证明学生反馈表是否可以真正提高教学质量。尽管如此,我认为反馈表可以作为指导方针,让我们每个人都更认真地思考自己的教学。积极的反馈结果会非常鼓舞人心,尤其是对寻求创新、另辟蹊径的教师来说。但如果许多学生都抱怨同一件事,并且几乎都给出相同的低分时,我们需要多加注意并采取相应行动。

也许我们需要回顾所教课程的实质内容,但我们可能更需要反思自己在课堂上究竟做了什么,以及为什么有些学生觉得我们未能有效地交流。学生们表达出的担忧可能包括授课中过于不正式的行动(如在教室中四处走动)、过于复杂或难以理解的PPT演示文稿、过多的材料被塞进一堂课中、或关于学生基本知识不切实际的假设。

几位同事告诉我,收到反馈表时,他们会有些恐惧,随后又会感到宽慰。而我的情绪变化并不如此。学生们对我在尝试的东西会作何反应?我一直对此感到好奇。就像毕翠克丝·波特(Beatrix Potter)笔下的塔比莎·特维切特(Tabitha Twitchett)夫人说的,有时我也会感到“受到了攻击”。但学生的反馈肯定是自我评估过程中的重要部分,而所有专业人员都必须参与。

学生的反馈有时会冒犯我们的虚荣心,但这是可以帮助我们更好地认识自己的宝贵资源。 

苏珊·巴斯内特(Susan Bassnett)是格拉斯哥大学(University of Glasgow)的比较文学教授,也是华威大学(University of Warwick)的比较文学名誉教授。


“律所会以客户评价来决定谁能成为正式合伙人吗?当然不会”

三十年前,学生对教学的评估(SET)几乎是不存在的。而如今,许多教师生活在对这些评估的恐惧中,且他们的恐惧并非毫无根据。尽管有证据表明SET是不可靠且有偏见的,但是它们仍被用来对付学者,尤其是临时性或在兼职合同下的学者。

需要澄清的是,我认为收集学生的反馈本身并不是糟糕的。我也并不渴望回到过去的美好时光——那时候教师可能在没有课程提纲和评分计划的情况下授课,甚至不需要对学生负责。但是,在决定聘用、续签合同和任期以及晋升与否的时候采信SET是一种错误的做法,并且应该停止。

首先,即便是非定量研究人士都会质疑这些数据的有效性。SET的回应率一般都较低,因此统计上的意义并不大。此外,由于很少消除异常值,只要有一个苛刻的评分,就能使结果明显偏斜。还有许多其它变量使得课程数据不可靠,而自从引入SET以来,这些变量明显扩大了。

大多数高校开始采用SET都是在90年代;当时的反馈表都是在一学期的最后一堂课上由学生亲手填写。而如今,任何学生都可以进行课程评估,包括几个月都没有来上课的人。不上课的学生通常对课程内容或教授的教学方法知之甚少;但是在新自由主义的学院中,他们得到了和其他“付费客户”一样的待遇,并拥有评论权。

Arrow
Source: 
Getty

我可以很武断地说,SET已被视为类似于Yelp的快餐连锁店评分,而不是严肃的学术行为。学生可以在任何时间、任何地点,以任何心态完成反馈表。我的学生们曾高兴地告诉我,他们是在半夜和同学聚会时完成反馈的。大多数学生自然没有意识到需要解决的问题,也没有意识到某些教师比其他同事有更多问题待解决。

作为系主任,我阅读过1000多份SET。根据我自己的调查,这些反馈表显然仍存在偏见。身为白人男性且自我认同为男性的人,不管是年轻的研究生还是年长的正教授,几乎总是被认为比我们其余的人更聪明、更具权威、更有趣,且教育水平更高。学生们是如此的性别歧视,以至于更偏男性化的女性学者也往往评分较高。

有此结论的不止我一个人。最近的几项研究都证明SET充斥着性别偏见;其中包括于2018年发表在《政治学与政治》(Political Science & Politics)的一篇引起广泛讨论的论文《学生评价中的性别偏见》。但种族偏见也很普遍。在英语国家,来自非英语背景的人也通常排名较低。

课程评价在刚出现的时候可能是个好方法,但这些评价很少被用于促进优秀的教学。通常,大家只在续签合同、评选终身教职和晋升时才会认真考虑这些因素。因此,一个人的职位越不稳定,SET就越重要。

鉴于学生反馈缺乏有效性,且充满有据可循的偏见,高等教育需要接受的是,不再依赖SET来做出重要决定并不是激进的提议,而是合乎逻辑的做法。医院是否让患者审查决定聘用哪些医生?律所是否允许客户评价决定谁能成为正式合伙人?当然不是。评论可以用来吸引新的患者或客户,但是没有人认为患者和客户最适合决定接受过高等培训的专业人士的价值。

同样的逻辑在高等教育也适用。在对SET的依赖消失之前,我们的招聘和晋升决定将继续受到根深蒂固的偏见的影响。

凯特·艾希霍恩(Kate Eichhorn)是纽约市新学院(The New School)文化和媒体研究系的副教授兼系主任。 


“我为自己提供了吸引学生的材料感到自豪,而得到能强化这种自豪的反馈令人鼓舞”

作为学者,我们非常习惯于接受反馈。我们的资金申请和论文稿件由编辑、小组成员和同行审议人员进行审阅,而他们的反馈经常十分尖锐。每当我们在会议上发表演讲时,总会有听众提出令人尴尬的问题。受到批评只是工作的一部分。但是,无论出于何种原因,学生的反馈都会给我们的工作带来特别痛苦的打击。

学生反馈对于某些教师来说是令人恐惧的。我会收到大量积极的反馈意见,以助力将来的晋升申请吗?还是说我会成为一个不喜欢上早课的学生的受害者?就我个人而言,学生对我的教授和组织的课程的反馈并没有让我受伤。我为自己提供了引人入胜且易于获取的材料感到自豪,而得到能强化这种自豪的反馈令人鼓舞。

例如,我教授大三生物化学课程的植物部分,该部分包括两个区块,即医学生物化学和植物生物化学。选修这门课程的学生中,有90%以上的人希望成为医生或研究人员。因此,我向学生们提供的内容超出了他们当前感兴趣的领域。但是,我认为这是让他们与材料进行互动的一次挑战。如果能让他们说出“这是我人生中第一次感觉植物可能真的很有趣!”这样的评论,那就是对我努力实现的目标最好的证明。

有时候,我在招收植物学研究生时,甚至会录取一些医学生。而这确实是很罕见的做法。

但是,并非所有反馈都是积极的,反馈也不应该都是积极的;而我们总可以改进。负面的反馈有两种,建设性和非建设性的。建设性的批评可能很有价值,我乐意接受。在我开设的课程中,有好几次我们都根据建设性的反馈意见更改了内容或评分标准。毫无疑问,这些变化对课程产生了积极的影响;学生们喜欢感受到自己的声音受到采纳,而非一味地在接受说教。这是双赢的局面。

Flowers
Source: 
Getty

然而,我的同事们曾收到的一些不那么具有建设性的反馈可能很难处理。“老实说你不应该讲课”这样的评论可能是一个玩笑,但是我怀疑写下这句话的学生是否考虑过这可能对教师产生什么影响。

我认为我们大多数人都在努力提供更好的课程——没有人会故意把课讲坏。换言之,尽管大家都在努力,但并不是所有人都有讲课的天赋。因此,这样的反馈可能会使人受伤。

全球很多高校都特别重视学生的心理健康。但是,学者们同样面临着保持良好心理健康这一挑战,而受到学生的言语攻击会对教师造成伤害。你能想象,如果情况倒过来,一名教师没有建设性依据地批评学生会造成怎样的骚动吗?

那么我们应该如何利用学生的反馈意见呢?就我个人而言,我强烈鼓励学生们分享他们对课堂的真实体验,不论是正面还是负面的。但是,我也提醒学生们,教师也是普通人,我们尽力提供良好的教学。因此,学生们可以提出批评,但需要具有建设性,并考虑到自己的话会造成的影响。

彼得·所罗门(Peter Solomon)是澳大利亚国立大学(Australian National University)小麦生物安全实验室的负责人。


“我的随机问卷与每个课程的结构和内容紧密相关”

最准确、最具洞察力、最真实的学生反馈无疑都认为麦克尔森博士是一个敏锐且善于动员学生的教师;整个班级都对现代早期意大利产生了长久的敬畏,并且学生们的智慧得到了可衡量的、永久性的提升。除此以外的反馈都是在说谎。

我显然是在开玩笑。但事实上,哪怕最好的反馈也不会达到上述幻想的程度。而当需要阅读学生反馈时,我们必须鼓起全部的勇气。

需要说明的是,一直以来都有研究反复证明,反馈表因不断重复社会对妇女有色人种偏见而对一些教师十分有害。比起了解教师,反馈表对于了解学生自己更加有用。

我所在的大学中,学生们的反馈意见通常专业且礼貌;他们避开了最严重的性别陷阱,并且避免发表带有个人情绪的评论。尽管如此,我们都对自己在课堂上的错误和失误有着充分的认识。我们预计学生们会像我们自我评判一样严厉地评判我们。在这种情况下,任何正面的评论都会像沙盒游戏《我的世界》(Minecraft)中的钻石奖一样闪耀;但负面的评论却让人感到更加真实。

如果你可以忍受,最好的解决方法是总结所有的反馈并写下来——这对申请工作也有帮助。这种做法迫使我们接受好消息和坏消息。

学生反馈的形式有两种。第一种是学校范围内的正式问卷,其中涵盖所有学科的标准问题。在针对大型堂课的调查中,这种形式最为有用;它提供了足够的样本量,并且能够在样本分布中显示异常值。有时,问卷会显示学生们花了多少功夫,他们阅读说明时有多仔细,甚至或许能表明他们对哪个话题最感兴趣。但是,调查结果也会明显表明学生很少注意到课程调查的内容和机制。

Secret
Source: 
Getty

我就职的大学的问卷中,曾要求学生在小组授课的调查中按姓名对每位教师评分,而学生们并不知道教师的姓名。我的同事开玩笑说,在我来到这所学校前,她曾是唯一能得到准确反馈的教师,因为她的名字是名单上唯一的女性名字。在进行反馈的那一天,我会和辅导课小组坐在一起,提醒他们哪个名字属于哪个教师。我列出了课程主题、头发颜色,如果需要的话,我还会说明年龄段和口音。这通常是可行的,但有一次,学生们一点印象都没有。不管我怎么描述这位杰出学者和受人尊敬的老师,他们都回忆不起来。最后,一个学生灵光一闪,叫道:“哦!他是在上课的时候说了‘他妈的’的那个!”于是所有人都恍然大悟并拿起他们的铅笔。

这种不稳定的表格在荣誉课程班级里并不那么适用。这些班级都很小,远远用不了等到学期结束,教师就会熟悉学生们的写作风格。而“课程评分方法使我能够证明自己的学习成果”这样的表述意义不大。这里我们就需要第二种反馈方式,即由教师自己分发,仅供自己阅读的非正式评估表。

与学校范围的表格不同,这些“游记问卷”与每个课程的结构和内容紧密相关。我的问卷要求学生说出自己最喜欢和最不喜欢的阅读材料,以及哪些主题应得到更多关注,哪些不应该着墨过多。问卷还探究学生从做演讲和听演讲中学到的知识,并就未来对课堂的调整征求意见。

我每年都会使用这些回答来进行真正的调整。与那些通常有害的一般性反馈表不同,这些反馈表向我展示了我的学生在哪方面读进了材料,而这常常使我惊喜。这也直接有益于未来的学生和未来的麦克尔森博士。

艾米丽·麦克尔森(Emily Michelson)是圣安德鲁斯大学(University of St Andrews)历史学资深讲师。


“课程评价在阐明事实的同时也在制造困惑”

在我职业生涯的早期,我曾教过一门莎士比亚研究课程。这是我一生中最好的教学经验之一。学生们很聪明也很投入。几乎每堂课我们都有很棒的讨论。我对自己的授课充满信心,而学生们也在创作良好且富有创意的作品。

课程进展得如此顺利,以至于在学期末,我对即将收到的反馈感到十分兴奋。作为一个处于职业早期的学者,我希望能得到高分,这将给我的教学档案添上一笔。因此,当那个棕色的大信封通过校园邮递送达时,我迫不及待地拆开了它。

然而,令我惊讶的是,我得到的分数相当平庸。在最后一个问题,“总体而言,你如何评价本课程”上,我得到的分数实际上低于本校平均分。我感到失望且困惑:到底发生了什么?

一种可能是,我对这门课的看法是错误的。也许我选择的阅读很乏味。也许学生们没听懂我的授课。也许我很无聊。

又或者我只是选择了错误的日子分发评估表。后来我意识到,我分发反馈表的那一天公布了期末考试的形式。也许对考试的一些焦虑掺入了反馈中。

直到今天,我仍不确定真相是什么;而这说明了课程评估的局限性。这些反馈理应帮助我们理解和改善我们的教学实践,但实际上它们在阐明事实的同时也在制造困惑。

因此,在上一个学期,我尝试在大一的一门课上进行了一个简短的“学习反思”作业。学期过去一半时,我要求学生以书面形式回答两个问题:

1)目前为止,你在这门课上学到的最有趣或最重要的是什么?

2)在剩下的半个学期里,你想在这门课上了解的一项内容是什么?

学生们的答案都很有启发性。课堂上不经常发言的学生表现出,他们其实对课程内容非常感兴趣。一些学生特别描述了从课程开始以来,他们的想法是如何变化的。其他人则指明了他们认为最吸引人的课题和仍不清楚的课题。

Jester
Source: 
Getty

这个作业用一种传统学生反馈表无法做到的方式帮助了我。它使我更好地了解我的学生。作为一名中年教授,我与学生的联系渐渐疏远。我所了解的流行文化已经过时。我的笑话有时候会冷场。我需要弥合与学生之间不断扩大的距离,以便更好地教导他们。

这种学习反思也为学生提供了与教授交流自身需求和兴趣的机会,并及时对教学进行回应。如果我们仅在课程结束并提交分数后才收到反馈,那我们应该如何处理这些信息?下一批学生会有相同的重视点和期望吗?

此外,一个关注学生的学习而非教授的表现的简短反思,相对不会引起针对教授的发型、着装、种族或性别的无用且有害的反馈形式。

综上所述,我们需要牢记,学生反馈并不只是关乎教师。这也是学生向教授追责的一种机制。尽管反馈可能被滥用,但当出现严重错误时,学生也有必要表达自己的不满。

因此,尽管其他征求学生反馈的模型可能对改善我们的教学实践更为有用,但传统的课程评估仍可发挥重要作用。

安德鲁·摩尔(Andrew Moore)是加拿大圣托马斯大学(St Thomas University)“伟大图书”计划的主任。


“与学生的互动已经开始让我回想起从事零售业的日子”

在每个学期结束时,我经常花时间与其他认识的少数族裔女老师交谈,一起读完各自收到的可怕课程评价。

每一轮评估都不可避免地带来侮辱性的评论,包括种族和性别偏见的重要证据。尽管我们已经知道数据显示,女性和教职员工的评分始终低于白人男性同事,但这仍然令人感到沮丧。

我最具挑战性的评估是在学期结束前一个月因产假离开,然后由我的助教接班。学生使用课程评估来抱怨我的产假,并批评这名助教(也是一名女性)“对待她的工作太认真”。我没有收到任何关于课程内容的有用反馈,而这次评分是我在教学生涯中获得的最低分。

有时候,课程评估中包含一些有用的想法,可用于将来的计划。然而,更多的是,它们仅仅是一个让学生分享对课程及导师主观感受的机会。这些主观感受从无关紧要的到不恰当的都有。对于大多数教师而言,当这些评估在晋升和任期过程中发挥核心作用时,这一点就很重要。

在各个年级中,女性和少数族裔教师的人数本已不足,学生评估可能会影响他们的归属感、对教学的信心以及专注于其他工作领域的能力。

课程评估也会对质量和严格性产生负面影响。研究表明,与被认为容易的课程相比,被认为更困难的课程始终获得较低的评价。尽管大多数教师不会在制定课程大纲时承认他们会考虑课程评估,但害怕受到负面评估常常会在确定课程内容和作业中发挥作用。

比方说,尽管我始终遵循大学发布的有关适当课业的指导方针,但我一再收到学生抱怨我分配的阅读量太大。结果,我发现自己每个学期都在试着减少分配的总页数,以避免投诉。

这个系统不可避免地会帮助形成所谓的“高等教育麦当劳化”。与学生的互动开始让我回想起在零售业的日子,而不仅是传统的教授和学生关系。交付服务的压力不断,要使学生感到像客户般满意。

可以肯定的是,学生应该有机会就他们的课程提供反馈。随着高等教育成本的飞涨,许多学生可以理解地更关注成绩、课程的“有用性”及其未来的就业前景。但是,鉴于不可能从他们的反馈中消除偏见的来源,学生评估在教师评估中只应扮演有限的角色。

系领导和行政人员应检查课程评估,看是否有严重问题的证据,比如教师失职。但是,继续采用它作为正式教师评估的一部分是不公平的,甚至有些不道德。

杰西卡·韦尔本·佩奇(Jessica Welburn Paige)是爱荷华大学(University of Iowa)社会学和非裔美国人研究的助理教授。

Umbrella
Source: 
Getty

“那些不会为眼下经历带来益处的行动,并不在学生们的议程之列”

学生反馈?如果有就好了。或者并没有。我真的不知道,因为我从中获益很少。

例如,在冬季学期末,我为300名学生讲了课。理论上,至少有300名学生:实际上,只有一半学生出勤。另一半,很显然在YouTube上以双倍速度看了我的教学视频,因此,我想他们的反馈可能是:“特里格宁博士听起来像是只花栗鼠,这有点儿令人反感。”

回到正题:原本应该有300名学生可以给予反馈,但我只收到一个答复。对此我的回应是只在一份学生论文上做标记并提供反馈,这对我来说似乎很合理!

说真的,我不知道自己为什么没能得到很多反馈,但这是一个问题。没有它,构成学术工作的所有其他内容的压力意味着修订讲义可能会落在待办事项清单之外。我已经会迅速地更新幻灯片,但当“过去的我”决定在其中一张关键幻灯片上用大红色字母写“更改此内容”会有所帮助时,“未来的我”会有时间来遵循指示。但是,“未来的我”并没有这样。

缺乏反馈也可能是职业上的问题。学生反馈是判断我们教学质量的一个标准,因此基于很小的样本量做出判断存在很大缺陷。它也可能也受到TripAdvisor的影响:只有那些有强烈意见的人才会提供反馈。

我们在学费时代所接收的叙述是,学生是消费者而不是教育伙伴。这不一定是我所看到的。根据我的经验,现在的学生几乎和过去的一样。他们可能对未来和求职途径有更多关注。但是他们仍然是在世上寻求占有一席之地的年轻人,碰巧在这儿听课。毫不奇怪的是,那些不会为眼下经历带来益处的行动,并不在学生们的议程之列。

关于缺乏反馈的最令人沮丧的事情是,如果收到反馈,它真的很有帮助。当然,在过去的10年中,我一直在接受一些精心选择的评价,我最喜欢的是说我的讲课“意外地令人愉快”。

当我开始教医学生时,我是不习惯面对大型演讲厅的。事实证明,把灯光调暗就是讲座即将开始的信号,没必要对那些闲聊和以2倍速播放视频的人大喊。我还保留了一些不够个性化的幻灯片,结果很快变得很明显——随后几年,好消息是,我收到的反馈使它变得更好。

但是,如果在我的演讲中没有什么别的东西可以反对或者称赞,那也是可以的。当然,这不是要把负面反馈作为个人成长的机会,毕竟我可以从我的孩子和Twitter获益的地方比这多多了。

约翰·特里格宁(John Tregoning)来自伦敦帝国理工学院(Imperial College London)呼吸道感染的高级讲师。

本文由陆子惠为泰晤士高等教育翻译。

后记

Print headline: At the sharp end

请先注册再继续

为何要注册?

  • 注册是免费的,而且十分便捷
  • 注册成功后,您每月可免费阅读3篇文章
  • 订阅我们的邮件
注册
Please 登录 or 注册 to read this article.

Reader's comments (12)

Everyone who is really interested in teaching and learning, that is teaching students on the basis of an understanding of the psychology, sociology and philosophy of education, would know that students feedback is always useful for improving our teaching of specific subjects and consequently what students learn. Indeed should be the case independent of what any professor (lecturer) would like to hear. Those who do not like student's feedback are simply not teachers because they have not been trained to teach i.e. Exposed to courses in psychology and sociology of childhood, adolescence and adulthood, lesson planning, rubric development, teaching practice, clinical supervision, assessment and evaluation, teaching methodologies. The key issue driving such aversion to students feedback is the fact that most university professors are lecturers. This who taught at lower levels of the education system or underwent some teacher training would never be subversive or would not be put off by students feedback. They would use that as an opportunity to improve their lesson objectives, content sequencing, learning styles, teaching methods. They would be welcoming feedback during the course knowing fully well that end-of-semester examinations is only one form of assessment/feefback.
In an ideal world, you might be right. Unfortunately, most university feedback forms are so badly designed, they are worse than useless. The list of faults with standardised feedback forms is long, but suffice it to say the feedback forms that I am forced to use (across three different universities): * Have statistically meaningless response rates. The vast majority don't fill them in, so how does it reflect the cohort's experience? * Rely almost entirely on Likert scales, often with 5 being a high rating on one question and 5 being low on another. The results then being summed to get an overall rating! * Have no context or actionable feedback (open field usually left blank). What do you do when some on the course rate a question 5 and other rate it 1? What do you change, if anything? * Are like TripAdvisor, in that they get filled in by people who either love the course or hated it. I suspect usually determined by the mark they got (You can tell from the few comments that are left). * Are often factually incorrect/just plain wrong in their description of the course and its delivery. I'm not talking about opinion or a different point of view, but objective facts e.g. Complaining about in-class tests when we don't do any. I am afraid I have lost all faith in the validity and usefulness of course feedback. It's a good idea and I wish I could get useful, actionable feedback, but the ways it is most often implemented makes it a waste of time. This is a reflection of the simplistic metrics driven environment within universities and the obsession with 'student satisfaction' rather than the quality of learning and education. The feedback forms are more 'Do I like my lecturer' forms, rather than 'Do I think I learnt something'.
Not all kinds of feedback are legitimate and helpful. To make the categorical claim that 'if all lecturers do not like student feedback, then they are not teachers' is simplistic. When looking at student feedback, there are questions about the truthfulness of the feedback (i.e., not all forms of feedback relies on postmodernist interpretation of truth; lies), and the qualifications on the person providing the specific feedback. Like what was reported in this article, students can feedback on the teaching and learning experience but they are not qualified to provide feedback on how learning or teaching should be done or what they should be learning or taught - because they are not qualified to do so. It is analogous to a patient giving feedback on how they were treated by hospital staff (legitimate) and how they like the surgery to be done (illegitimate). There is simply no evidence that student feedback reflects reality. Rather than just making claims - show me the evidence for its veracity.
There is always one: one student who has to write something abusive; one student who gives a feedback grade of 1 out of 5 across all the questions because of a mark awarded they felt was too harsh; one student who has to write that, "the lecturer is past his sell-by-date"; one student who gets his/her lecturer mixed-up with someone else and hurls abuse at the wrong person; one head of department who sacks lecturers because their course evaluations are below the mean for the third year in a row; and one reader of commentaries like these who has no empathy for those lecturers for whom unjustified negative feedback from students can be both health and career destroying. Formal student evaluations are so generic in their design that they are incapable of producing much that is a positive help to quality improvement. On the other hand, they do give the customers a sense of power and an opportunity for payback for perceived wrongs, real, imagined, and misunderstood, but is that why we use them? Informal student evaluations, such are described in these commentaries, are a positive boon to any teacher. Quite how they morphed from informal to formal is beyond me, though i do remember vividly the introduction of formal student feedback in the early 90s: being denied permission at that point to continue to use my own customised feedback instrument, one that did actually help me greatly for the four years i used it. It is time to abandon this destructive formal feedback mechanism and replace it with an informal one that all staff must use, such as the two-question version presented in these commentaries. Then, student feedback will mean something positive for all which, as all "trained" teachers know, is why it is was sought in the first place.
It's a shame that this article did not start with a good review of the research on SETs. Most Universities do not use validated instruments which is far from ideal, most are knocked up by a committee, which results in a camel. By validated feedback instruments do exist and they should be used and researched more in-depth. They are not perfect, but then no measurement instrument is, particularly those that measure social interactions and their impact. I am however struck by how even the outcomes of rough and ready evaluation questionnaires tend to match what Faculty already know about each other.
Can you please post some examples here?
Student evaluations replicate the infantilising scores or gold, silver, bronze, tin "stars" so commonplace in academia - and beyond. While the comments can be really useful, I really don't know anyone who believes that knowing they have scored 3.86 (or 4.24 for that matter) overall is in any way informative. The comments can shine a light on things one should rethink when they're about what worked, what worked less well, etc., but all too often there is always some student more interested in being spiteful or finding their most personally offensive put-down as a chance to express their frustratio, rather than engaging with constructive criticism. If we gave feedback of the sort some of us have received (and yes, I have also received some gratifying comments!), we would probably be had up before some committee or other. The idea that evaluations are anything else than management tools is disproved by the fact that few have been called in by their line manager to be commended for their teaching. But if the scores are poor, you'll be the first to know! If these were designed to be helpful, scores would be dropped leaving only comments, with someone having gone through beforehand to remove all offensive or needlessly unpleasant comments. Innovations in pedagogy jar with ways with which students have become comfortable and, by definition, anything that jars risks producing senses of disorientation. These can be incredibly transformative for students who throw themselves into things, but the unfamiliar can equally be a source of anxiety. Risk taking for its own sense is foolish, but if one's approach to teaching is to transform your own and students' relation to the world, then it is all important.
Law firms do let clients determine who makes partner. Not by reviews, but by number of clients and how much they have been willing to pay. Literally if you don't have clients and large billings you do not make partner. I don't think this would be what most professors would support.
A lot of the flaws in SETs pointed out in the article and the comments can be designed out if there is concerted collaboration between the SET administrators, academic leaders and representative teachers and students across each discipline. Some universities chose to make this investment, some do not. Here is a starting list of what can be 'fixed': shifting emphasis in questions from teacher 'performance' to student learning experience; periodically updated and validated instruments; not running the standard SET when a teacher is trialing a new approach; running some form of mid-semester feedback (either formal or informal); regularly reminding students of their obligation to provide constructive feedback and potential consequences if they do not; providing students with a free-text field for each item not just the overall satisfaction item; dealing with low response rates in reporting of results (e.g. not reporting quantitative results if a statistically valid response rate threshold has not been met; use moving averages over multiple semesters); analysing student feedback for your university to determine the actual level of gender or racial bias to inform localised responses; taking the time to tell students what has been changed in response to their feedback and that of previous cohorts; inviting students to co-design improvements such as group tasks, assessments, etc; having a clear policy to distinguish how feedback on the course/subject vs. feedback on the teacher will be used/not used. While SETs will always be an imperfect instrument, with a bit of effort they can be turned into something approaching fit for purpose.
Thank you for this post it really helped me. Keep it up. https://islamiclives24.blogspot.com/2020/04/prophet-muhammads-love-for-children.html
Two stories and one observation. In the days before standardized university issued student feedback forms, I issued my own short and free response only questionnaire to part time students on a masters course, who were all employed senior managers. Given the block release delivery and high price of the course, I used a local hotel. One individual on an early cohort consistently offered the comment that 'the coffee is awful' at the end of each residential workshop. I consistently responded that his subjective one out of 30 plus opinions was insufficient for me to take any action. The second story related to a similar course, this time designed and delivered in-house to a group of middle managers employed by a large corporation. I was asked by my then Dean to replace a colleague to teach a particular module because the colleague had received negative feedback from the students, passed onto the Dean by the corporation's Management Development Manager. I asked my Dean, ''Do you want me to get consistent (high) scores of 5 on the student evaluations, or do you want me to do my job of educating and developing these managers?'' He thought for a moment, I think about whether to challenge my cynicism, and then replied, ''Get a positive evaluation; we need the business from this client''. I did just that. My observation is hinted at in some of the above. It is simply what grounds are there for believing that changing something to satisfy this group of individuals will help satisfy a new and different group of individuals? As my first story indicates, I believe collecting feedback is useful and should be done. But always tailored to specific courses. And student evaluation itself always has to be evaluated.
I encourage diaglogue with students throughout the module, and am always ready to listen to (but not necessarily act upon) their comments... but this is a habit developed through some years teaching in FE before slithering into a university. I do sometimes wonder at the questions that are asked on the evaluation forms that the students are sent, and the over-reliance on Likert scales rather than getting them to say what they think. OK, it's easier to use metrics as an overview, but they are not very informative or helpful when you are looking to improve.