Grades Gone Wild: Revealing the Relationship Between Grades and Student Evaluations of TeachingPosted: August 29, 2012
BY Bert Smoluk, Ph.D.
Imagine that you are part of a large sales force and the guy three doors down the hall just won the annual salesmanship award for the third time. Customer satisfaction surveys show that his customers love him, he takes excellent care of their needs, and unit sales are phenomenal. This salesman is due for another hefty raise.
The rest of the sales force, and you, agree that this guy has sales talent and an increase in salary is appropriate. Soon after the award you are having lunch with a customer of the salesman who outperformed everybody. This customer tells you that she gets all her products from this great salesman at little or no cost! After choking on your ham sandwich, you regain your composure and find out from the customer that she is not the only one receiving free goods. Most of the customers of this salesman receive free goods.
Soon after lunch, you investigate this issue with the accounting department of your company. It turns out that the accounting department is antiquated and simply does not have a clear record of the star salesman’s revenue. The only thing the accounting department knows is that the salesman oversaw the shipments to customers for hundreds of units. After an intensive investigation and digging it is revealed that this salesman has almost no sales revenue! He has been giving away the store for years.
Sounds too bizarre to be true? Surely it can’t happen in your organization. As strange as this story sounds, this type of behavior is common place in academia. The goods given away are grades, to be more precise, high grades, and there is no accounting department. It does not exist!
This blog is about why there should be an accounting of the relationship between student evaluation of teachers (SETs) and the grades awarded by instructors. My argument is that without such an accounting instructors will give higher grades in return for higher SETs. The research supporting my argument is based on the literatures from economics, psychology, ethics, and education. Economists come into the picture because there are financial benefits, with no apparent costs to both parties involved: students can translate higher grades into higher salaries and professors can translate higher SETs into higher salaries. Psychologists are interested in the field because there are some subtle, yet powerful, dynamic interactions going on in the classroom between students and teachers that depends on the situation and the motives of the parties involved. Moral philosophers are interested because there are ethical questions about the possible quid pro quo behavior of teachers inflating grades and students, in return, giving favorable SETs. Educators are interested because they want to know how to become more effective teachers. They recognize that some faculty may be inflating grades because they are under pressure to demonstrate their teaching skills, see Matos-Diaz and Ragan (2010). Educators are also concerned about the dysfunctional behavior, such as low expectations on the part of the instructor, caused by heavy reliance on SETs as discussed in Crumbley, Flinn, Reichelt (2010). While this entire blog is about education, the following discussion focuses on areas of economics, psychology, and ethics.
It does not take much imagination to realize that higher grades can translate into higher expected income for the students upon graduation. Many prospective employers require a minimum GPA (grade point average) just to be eligible for the first job interview. A GPA sends signals to prospective employers about an individual’s motivation and potential job performance. Thus, not only are higher wages more likely with inflated GPAs, but job search costs including the opportunity cost of transitional unemployment, are reduced. Many employers expect to find a recent college graduate’s GPA listed high on their resume along with their major and alma mater as it represents crucial information in assessing an individual’s capabilities for many jobs. Employers often request transcripts that list every course taken and grade received by a job applicant.
While it is obvious to any causal observer that higher grades will help open doors and benefit students, the subject gets more interesting fast. According to Babcock (2009), higher grades translate into reduced learning as students spend less time studying. His research shows that the average study time in a course in which the average expected grade is an “A” is approximately 50% lower than a course taught by the same instructor in which the average grade expected was a “C”. Students, often juggling many classes and responsibilities, are pressed for time. As a result they are attempting to balance the marginal cost of time spent studying in a course verses the marginal benefit of the expected course grade. Courses with inflated grades disrupt this balancing act and causes students to focus their efforts on other courses where the marginal benefit gained from studying is higher.
Thus, it is not hard to see from an economic perspective that higher grades can lead to higher SETs as students reward instructors for higher expected future pay, reduced work load, less stress, and more time saved.
Even if we ignore the economics of the situation, why would students reward teachers who give higher expected grades? After all, teaching evaluations are supposed to measure the quality of instruction and the ability of the teacher to communicate effectively difficult subject matter. Students’ opinions on teaching are supposed to be unbiased of any quid pro quo relationships. This is where psychologists enter the picture.
The empirical work of Matos-Diaz and Ragan (2010) shows that teachers with lower variances in their grade distributions obtain higher SETs. They reason that grade inflation compresses the grade distribution of the class (towards “A”s) and thereby lowers the uncertainty about a student’s expected grade. Students, like most people, are risk averse, they prefer less uncertainty in their life. In fact, there is a long line of research in behavioral finance that shows individuals prefer early resolution to uncertainty rather than later resolution. In the subject at hand, this means students prefer teachers who are known to give inflated grades. Students find comfort early on in the semester knowing they face less grade risk in an inflated course compared to a course where grades are more normally distributed and the final outcome is less certain. Thus, students reward instructors who make their lives easier and more comfortable.
Ethical issues abound in the study of grade inflation and SETs. Crumbley, Flinn, and Reichelt (2010), for example, state that emphasis on SETs by universities has led to unethical behavior on the part of professors who inflate grades and deflate course work to gain favorable teaching evaluations. Such unethical behavior, they claim, is akin to executives “cooking the books” through inflated earnings. Still others argue that grade inflation unfairly favors students who actually underperform in a course relative to those who exhibited strong perform. In other words, better performing students are devalued with grade inflation and underperforming students are over-valued.
Some Final Thoughts
Educators know that employers look closely at a job candidate’s grades. Grades are supposed to reflect student performance in a course. If the course is taught at a reasonably challenging level, as many employers expect, some resemblance to a bell curve distribution of grades should result. This distribution comes about because students come into the class with varying skill sets, motivations, and interest. Most of all, effort towards the course may vary significantly. Employers rely on the university and its professors for an honest assessment of a student’s performance. Grade inflation undermines the integrity of the institution.
There are several ways to combat grade inflation. One is to adapt a transparency approach that publishes the grade distribution of each course taught at the institution. Examples are readily found on the web, see for instance, The University of Wisconsin and Texas A & M University websites, http://registrar.wisc.edu/course_grade_distributions.htm and http://registrar.tamu.edu/facultystaff/report/default.aspx, respectively. A second approach is to develop targeted grade distributions before a course begins. A third approach is to reduce the emphasis on SETs in evaluating instructor performance. A detailed discussion of these remedies, like nearly all of the issues addressed here, could easily be the focus of another blog.
Empirical studies that compare SETs to grades are notoriously plagued with data problems, making inference about SETs and grades very difficult. Across classes, there are vast differences in student ability and student experiences in past and contemporaneously course work load, and differences in the expected grades in other classes taken in the past and concurrently. SETs are also known to vary depending on class size (Bedard and Kuhn (2008), required verses elective courses, and the quantitative requirements of the course. Controlling for these attributes is difficult.
Nearly all the research I reviewed for this blog recognizes that grade inflation is systemic in the United States. Some authors, Babcock (2009) for instance, recognize that an alternative to the negative interpretation of grade inflation is that students are better prepared. Not surprising, many of these same authors find this alternative explanation lacking substance, since most generally accepted standardized aptitude tests show no such improvement.
Babcock, Philip, 2009, “Real Costs of Nominal Grade Inflation? New Evidence from Student Course Evaluations,” Economic Inquiry, Vol. 48, (4), 983-996.
Bedard, Kelly and Peter Kuhn, 2008, “Where Class Size Matters: Class Size and Student ratings of Instructor Effectiveness,” Economics of Education Review, 27 (3), 253-65.
Crumbley, Donald Larry and Ronald E. Finn, 2010, “What is Ethical About Grade Inflation and Coursework Deflation,” Journal of Academic Ethics, 8, 187-197.
Darby, Jenny, 2006, “The Effects of the Elective or Required Status of Courses on Student Evaluations,” Journal of Vocational Education and Training, 58 (1), 19-29.
Matos-Diaz, Horacio, and James F. Ragan, 2010, “Do Student Evaluations of Teaching Depend on the Distribution of Expected Grade?,” Education Economics, 18, (3), 317-330.
Dr. Smoluk is Associate Professor of Finance in the School of Business at the University of Southern Maine.
(NOTE: John Voyer posted this entry on behalf of Prof. Smoluk)