Teaching Council Report on SAIS
29 October 2003
Baldwin Lee, Teaching Council Chair
The Teaching Council has met twice this semester to discuss the impending conversion from the current teaching evaluation system, which is paper-based, to an on-line electronically-based system. We are grateful to have had the full cooperation of Donald Scroggins, Director of Institutional Research and Assessment, and Julie Little, Executive Director of the Educational Technology and Innovative Technology Center.
The issues the committee discussed centered about two major topics. First we explored the process, and its attendant consequences of the conversion of SAIS from paper to on-line.
Sustaining the statistical validity of the evaluations will be a major challenge. Prior studies have shown that when the means of conducting evaluations is on-line, the percentage of participation drops precipitously. The current paper-based system has a participation rate of approximately 80% whereas the on-line version participation rate is only 30%. It was agreed that our goal was to have this participation rate match or exceed that of the current paper-based process. This drop off is attributed to the voluntary, outside of class procedure that is engendered by conducting the evaluations on-line. The current system is one where evaluation forms are completed during a regularly scheduled class. A concern stated was that the probable motivation for those students who would voluntarily participate in an on-line process would be to utilize the opportunity to either register condemnation or praise. This minority of students would probably represent to the extremes of the bell curve leaving absent the views of the majority.
The committee discussed a range of possible solutions to this participation problem. Solutions seemed to divide themselves into two categories: incentive-based or penalty-based. It was much more difficult to devise "carrots" than "sticks". In the "stick" category solutions proposed included withholding grades and suspending access to on-line services. The sentiment of the committee was that it would be far more preferable to find "carrots" rather than subject students to what could be perceived as institutional tyranny. Unfortunately, finding incentives is more vexing.
The drop off of participation in the on-line evaluations caused the committee to explore reasons for this phenomenon. A presumption was made that many students consider to evaluation process, without regard to its format, as largely irrelevant. Despite the fact that Tennessee 101 is used by many students, there exists a sense of futility and frustration that pervades the procedure. Many students feel that there are no tangible benefits from their participation. There is a widespread view that some instructors, whose ratings are low semester after semester, show no improvement. The cause of insufficient student participation was deemed a product of lack of relevance of the evaluation rather than the format through which it is conducted. Many felt that evaluating an instructor at the end of a class offered no benefit to themselves.
A meeting with several student leaders from the SGA provided some insight into this matter. It was proposed that a "carrot" that would encourage voluntary student participation would be based upon providing a sense of empowerment to students. A proposal was made to allow for students to provide an instructor input at some point during the course as well as having the overall evaluation occurring at the end of the semester. If this student input were to occur at mid-semester, students would have the sense that the observations they conveyed to their instructors gave them a voice that could make their class experience more significant.
Some faculty members were less than enthusiastic about this proposal. Their reasons included questions about the ability of a student to make a dispassionate judgment of an instructor's methodology and pedagogy. There were also opinions expressed that mid-semesters evaluations would have questionable validity because judgments would be made based upon incomplete data. Faculty members raised issues that have been voiced since campus-wide teaching evaluations were first initiated. These included challenges as to what was being measured, the legitimacy of purposes to which the data would be used, those who would have access to this data.
It has become clear the scope of the task of the Teaching Council in this matter of evaluation of teaching is broader than previously envisioned. Devising mechanisms to assist in the conversion of the paper-based system to one that is on-line will not address the principal concerns of those involved. Our agenda must include a review of the following:
Establish a philosophical common ground for concerned parties
Determine what will be measured
Define the purpose of evaluations
Ensure the best use of the resources of the Office of Institutional Research and Assessment, OIT, ITC for the conversion to an on-line process
Cite how the collected data will serve instructors, students and others
Propose guidelines as to how this data will be utilized
The potential for success in achieving these goals is predicated on the assumption that there exists a common ground where the interests of faculty members, students and others intersect. The Teaching Council recognizes that the many fundamental concerns of each of these constituencies are diametrically opposed. It is there our task, and the task of all those concerned to explore the potential of policies and procedures which can serve the common good.
An initial step may be useful is to assert that the greatest value of the Student Assessment of Instruction System is to provide information of value to instructors and students. It may be useful to eliminate or change the word "evaluation" because of its obvious negative implications. Although an instructor might challenge the authority of students to engage in an "evaluation" of his/her teaching, most instructors would value some form of input from their students. The challenge to the faculty is to determine what it is they wish to learn from their students. Students must be provided with incentives to encourage the fullest and most thoughtful participation possible. Making appeals to their civic responsibility to the University community is insufficient. The challenge to the administration is to provide the support and the means to implement the suggestions that will emerge. A further challenge to the administration is to affirm that the purpose of the SAIS is to improve the quality of the educational environment for students and faculty and to assert that the use of collected data for punitive purposes undermines the trust necessary for this process to succeed.
The Teaching Council is calling upon all faculty members and all students to submit their responses to the challenge to make the SAIS procedure relevant and fair.
