Evaluating Your Teaching

Evaluation is the unsung hero of educators and it is arguably the most important step in the education life cycle. Without it, how would you know if you teaching is working? How would you know if your students are learning? Asking the right questions will help ensure that you are doing all you can to help your students achieve their potential. This article serves as an introduction to evaluation for educators.

“The key purpose of evaluation is to enable us to gather information that gives a measure of the effectiveness of the training we deliver. We can then use this information to improve future sessions or programmes that we conduct.”

Francis & Gould 2000

Questions to Ask

Rather than diving straight into the methods of evaluation, let’s have a look at the questions you ought to be asking. Picking the right questions will provide you with answers which can help continually improve your students' learning experiences. The key is learning how to get feedback into your learning design!

Does the lesson work?

What are they learning?

Can they do the right things?

Are they doing the right things?

Does the Lesson Work?

Begin by evaluating your lesson plans. The most important question you can ask yourself is “Does it work?”. This is a broad question and can be broken down into five smaller domains.

Do I have enough or too much content?

Are instructions clear enough so learners know what to do?

Do the lesson timings work?

Are learners engaged or are they bored?

Are learners keeping pace?

The most effective way to answer these questions is by observing learners during your lesson. Close observation will reveal any snag points and you will be able to make adjustments to improve the overall experience. You may find a pilot test group is an excellent opportunity to explore effectiveness before wider adoption. An alternative option is to use a survey to gauge the student reaction to your lesson. See the section Methods for Evaluation for more information on both techniques.

What are they Learning?

Suppose your lesson plan is working well and everything you are teaching is retained by your students (I know, this is an impossible feat!). The next question you should ask is “Are the students are learning the right things?”. This question refers to both the subject matter and the level of learning required from them. Here we focus on the latter aspect.

The required level of learning is subtle and refers to the difference between recall and recognition. Recognition requires application of basic facts to a situation while recall requires a greater in-depth appreciation of the subject material and is more difficult to achieve. You should choose the required level of learning based upon how the students will need to use the knowledge in the real world. The most effective way to answer these questions is by testing your students. So, what level are you teaching your students at and how should you test for each?

Testing for Recognition

This assesses whether your students are able to recognise concepts, but not necessarily recall them. Recognition is best assessed with scenario-based questions that provide multiple correct options of varying appropriateness. The most appropriate answer should score the most points, while less favourable (bit still plausible) answers should score fewer points.

Testing for Recall

This assesses whether your students have grasped the taught concepts and can explain them to another individual. Recall is best assessed by a human judge and can be achieved through white space questions or through observed performance.

A quick word on multiple choice tests. They are the most common type of formal assessment because they are advantageous to teachers; these questions are efficient to write, administer and score. Furthermore, they provide a consistent and objective grading. However, multiple choice tests offer few, if any, advantages for the learner. If you use them, please ensure you follow the advice of providing answers of varying appropriateness rather than using simple correct and incorrect choices.

Can they do the Right Thing?

If you are assessing recall knowledge or the acquisition of skills, you will need to observe your students performing in order to evaluate them. The process is actually a cycle with the final step being the provision of feedback. The difficulty with this type of assessment is that there is not often a single ‘right’ answer; consequently, use of multiple assessors will lead to inconsistent grading. You should consider these three aspects for any performance-based assessment:

Who will give the feedback?: Do they have sufficient expertise in the subject material AND are they experienced at evaluation and feedback? As well as expert reviewers, you might consider peer-to-peer reviewers, near-peer reviewers or self-evaluation.
How can you make the feedback valuable and consistent?: The more subjective a performance is, the more the feedback will vary between assessors. The variation could be mitigated by using a checklist or a rubric.
What are your learning objectives?: Ideally you should determine the evaluation method before you create the lesson plan. This is possible by using your well-crafted learning objectives. The result is a clear road map for designing the lesson, as well as reassurance that you are teaching at the right level.

Are they doing the Right Things?

This is often poorly measured. It tells you if your teaching has been successful in implementing a long-term change and the learning is now being used in the real world. It can be assessed in multiple ways, but often the most accessible way is to compare organisational performance before and after teaching. This can be done with a control and study group to reduce the effect of external events on the outcome (i.e. A/B testing).

A further option is to observe the students’ new practice and see if a change has occurred. Teachers may choose to directly observe a small cohort. Alternatively, you could monitor this indirectly via supervisors or through monitoring of signals such as incident rates or productivity measures. A third option is to use qualitative interviews and case studies. Ask trainees to tell stories about how their practice has changed or perhaps describe difficulties they continue to face. The greatest insights arise when learners who have benefited the most are compared to those who have benefited the least. These learners can be identified by using a short follow up survey sent to all students.

Methods for Evaluation

When planning your evaluation, you should take a moment to consider what you would like to know and how you will find out the answer. Also consider who you will ask and when you will ask them.

What?

How?

Who?

When?

Formal Evaluation Options

Methods for evaluation can be broken down into four broad categories. We will discuss each in turn.

Self-Monitoring:

This involves asking yourself (or completing a short self-evaluation form) whether you have met your pre-determined objectives. It allows you to consider what went well and if there are any aspects that might be improved next time around. You can read more on reflection in this article. While you will strive to be fair and objective, this process does involve self-judgement and some of your biases may lead to misinterpretation of events.

Audio or Video:

This involves recording your sessions and reviewing them with other colleagues to identify areas for improvement. They provide objective evidence of what is actually occurring in the lesson and enable teachers to monitor their own performance without the mental burden of teaching at the same time. For best results, you should review the recordings with colleagues to identify a variety of strengths and weaknesses. Repeating this process enables you to track progress over time. Remember to get consent from the students.

Colleague Feedback:

This involves asking other colleagues to sit in on the lesson and directly observe teaching as it is being conducted. They should then give feedback and constructive criticism. Before the lesson you should negotiate the areas of focus. You might be observed by peers, senior colleagues or external professionals.

Student Feedback:

This involves seeking the students’ perceptions of the lesson and it can be used as a marker of how effective they are learning. You should regularly seek feedback so you can correct any poor practices as early as possible. This may be done formally or informally. There are three common methods:

Surveys

These provide a snapshot of how favourably learners react to your teaching. Prioritise what you want to find out, try to keep surveys to less than four questions and use at least one open ended question. Consider using a Likert-scale.

Advantages: They can be efficiently administered to a large number of students to gain a comprehensive picture. Free expression is promoted by anonymous responses. Negative responses are useful for highlighting problems.
Disadvantages: Questions are standardised, and teachers cannot probe the answers further. Furthermore, the results might be influenced by students who wanted to be nice or because they didn’t really understand the content. At a deeper level the reality is that surveys don’t tell you if the teaching really works. William Thalheimer has explored this issue and found that survey responses may not directly relate to actual behaviour. For example, students may be pleased with the teaching, but they have learned little. Read more in his book "Performance-Focused Smile Sheets".

Interviews

These can be performed with focus groups. Interviews can be led by the teacher or by an external individual. Set questions that you are interested to know more about and make sure you probe students for more detailed clarifications.

Advantages: You may make some unanticipated revelations. You are able to probe and further explore the reasoning behind the expressed opinions.
Disadvantages: The views may not be representative of the larger group as only a few students are involved.

Test Results

The effectiveness of your teaching can be assessed by administration of assignments.

Advantages: Provided students are asked questions about a given topic, assignments will tell you what they have learned and what they have not.
Disadvantages: They do not tell you which teaching methods led to better student learning.

Informal Evaluation Tools

Students are the easiest source to gather feedback from. Aside from surveys and interviews, there are a few short tools you can use to gather feedback during the lesson. Remember to vary your methods to prevent students from getting bored. Five methods are described below.

Minute Paper

At the end of the lesson, ask your students to answer variations of the following two questions. Students should then write their response on a piece of paper and hand them in.

“What was the most important thing you learned during this class?”
“What important question remains unanswered?”

Muddiest Point

At any point in the lesson, ask students “What was the most confusing point in…?”. They should either hand in their answers on a piece of paper or write them onto a whiteboard for the class to see. You should then respond and clarify these points.

One-Sentence Summary

Ask your students about a given topic “Who does what to whom, when, where, how and why?”. They should then create a single long summary sentence. This can either be shared with the class or collected to review at a later date.

What’s the Principle?

Create several short problems which illustrate the core principles of your lesson. Next provide students with the problems and ask them to state the principle that best apply in order to find a solution.

Traffic Light Feedback

Use red, yellow and green sticky notes. Ask students to write down which methods or learning or aspects of the module they:

Would like to reduce (red), continue (yellow) or see more of (green).
Don’t understand (red), like more discussion (yellow) or fully grasp (green).

References

Design for how people Learn; Julie Dirksen
University of Hong Kong: Evaluating Your Own Teaching
Vanderbilt University: Classroom Assessment Techniques