Books on bookshelf

Research on Evaluation of Teaching: Student Ratings

The Office for Faculty Advancement has developed methods to help faculty members collect both detailed mid-term feedback from students and/or immediate feedback at the end of a class. You can also do a self-analysis, arrange to have someone observe you while teaching and give you feedback, or watch yourself teaching on videotape.

Mid-Term Feedback on Your Teaching

The Academy has designed a number of processes faculty members can use to collect detailed and candid reactions from students. One such procedure, called a Student Small Group Analysis (SGA), allows faculty members to receive these reactions long before the term is over, providing information and insights that often do not emerge from end-of-term student ratings and comments.

This simple procedure will provide you with early and extensive feedback from students. Best of all, you do not have to wait until the semester is over to receive feedback. If changes are appropriate, you can make them for the same class that made the suggestions.

An Academy representative comes to your class at a time and date you determine, usually between the third and sixth week of classes, and takes up the last twenty minutes of class time. You leave the room. The Academy representative will then say something like the following:

My name is XX. I’m here today at the request of your instructor to do something we frequently do for people who teach at Montclair State. I’m going to collect some information from you on how this class is going. Your instructor is interested in your ideas and wants to hear from you. I will share with professor X what you tell me, but I will protect your anonymity entirely. Please be candid

The Academy representative then divides the class into relatively small groups, and asks each group to spend eight to ten minutes discussing these three questions: In what ways has the instruction/instructor helped you learn in this course? Can you suggest some changes in the instruction/course that would better help you learn? Are you learning? Why or why not?

Each group receives a note-taking form with the questions listed above, and one of the students take notes of the discussion that follows in that group.

The creation of the groups tends to promote the development of ideas and a level of insight that seldom, if ever, emerges when students operate alone to scribble a few comments on the back of a form at the end of the semester.

At the end of the allotted time, the consultant brings all of the students back together and begins to get reports from the groups. Here’s how it works: First, the consultant picks one of the groups or pairs and says, “I’d like to get this group to give its response to the first question. Please listen to their responses so you can tell me if you have any additions or disagreements.”

At this point, the consultant can do something you cannot do with written responses. The consultant can clarify, asking follow-up questions, asking for examples, asking the questions we all want to ask when we read student comments on student rating forms. Second, the consultant quickly asks for any additions to or disagreements with the first group’s responses. At this point, the consultant is able to verify, finding out whether others share the views of the first group (or pair).

On written evaluations, we tend to think that if five or more students make the same point, everyone must agree. The SGA often demonstrates otherwise. Students do disagree, sometimes strongly. Thus, the airing of ideas can be as revealing to students as it is to the professor.

Third, the consultant picks another group to start discussion of the second question, and if time allows, still another for the third. The consultant takes notes on the discussion, collects the students’ notes from each group, and prepares to meet with the instructor to share the results.

Numerous faculty members take advantage of this service on a regular basis. It is one of the quickest, easiest and most effective things you can do to improve student learning in your classroom. And while we can’t guarantee higher ratings for you, we have found that students tend to be highly appreciative of instructors who make the effort to get student input on the course’s progress, and that changes made based on their input can improve the class dramatically.

The results of a Small Group Analysis are strictly confidential and will be shared only with the instructor, and no one else.

Collecting Immediate Feedback on Your Teaching

To collect immediate feedback (IF) from students at the end of a class or other session, many faculty members are using a simple procedure first used in 1966:

End the class five minutes early and ask students to respond to two questions:

1. What major conclusion did you draw from today’s session?

2. What major questions remain in your mind?

The process of answering these questions will (a) help students to ask themselves extremely valuable questions (what have I learned; what do I need to learn now) and (b) provide instructors with enormously valuable feedback. Sometimes instructors discover that students are drawing conclusions quite different from the ones intended. Some instructors begin the next session with responses to the patterns that emerge or make adjustments in the way they teach or in explanations they make.

Some professors like to add an additional question occasionally, or substitute it for the second question:

3. Why did you draw that conclusion?

That question can also stimulate students to think more deeply about their own thinking and why they think what they do.

Traditionally, students have submitted their responses on paper, often with no names attached. In recent years, some professors have occasionally asked students to submit their responses online, using something like the Nicenet site to do so.

Self Analysis

The Course Analysis Project focuses on a series of questions on “How do I plan my courses?” and encourages faculty members to use those questions to examine the courses they teach. When we ask outstanding teachers what questions they ask themselves as they prepare to teach a new course, we usually hear some combination of the following:

What questions will the course help students answer, or what abilities, habits or qualities will it help them develop? What reasoning abilities must they develop to answer these questions? What information will they need and how will they obtain it? What questions will you ask students in order to focus their attention on significant issues, or to clarify concepts, or to highlight assumptions that they are likely to ignore? What writing will you ask them to do that will help them grapple with these matters? How will you confront them with conflicting claims and encourage them to grapple (e.g., collaboratively) with the issues? How will you find out what they wish to know? How will you find out how students are learning and provide feedback before and separate from any grading of the students? How will you help students learn to learn, to examine and assess their own learning and thinking, and to read more effectively, analytically and actively? How will you keep students thinking when you communicate with them?

In short, how will you create a natural learning environment with the skills and information you wish to teach embedded in assignments (questions and tasks) students find fascinating–authentic tasks that arouse curiosity and challenge students to rethink their assumptions and examine their mental models of reality? How will you spell out explicitly the intellectual standards you will be using in assessing their work and why you use those standards? How will you help students learn to assess their work using those standards? How will you know when students are able to do what you want them to do intellectually?

Videotape Your Class

Faculty members can arrange for the Office for Faculty Advancement to videotape a class. This service offers faculty members and graduate teaching assistants an opportunity to examine their teaching closely, whether in the classroom, laboratory, lecture hall or studio. A student technician, positioned in the back of the room, records the class using a small unobtrusive video camcorder and a wireless microphone. The technician usually focuses on the instructor but can also capture a record of student actions during class if instructed to do so. At the end of the class, the technician gives the recorded cassette to the teacher (ensuring that the professor will have complete control of it), along with a viewing guide prepared by the Office for Faculty Advancement. The instructor can ask a consultant from the Office for Faculty Advancement to view the tape, seek feedback from a colleague or view it alone. Instructors can also arrange for the Office for Faculty Advancement to videotape a practice lecture or discussion. Call the Office for Faculty Advancement for details.