Using A2L Items
Many teachers who try a formative approach like it very much and feel that students benefit a lot. Unfortunately, they then find themselves unable to cover the required or self-imposed syllabus. Using items is pointless if one rushes through them, foreclosing the ability to think and discuss all the ideas students have manifested. In fact, superficial use of items, where the correct answer is given and the item discarded, is worse than not using the item at all. Such a shallow use only reinforces the idea that the correct answer and not the reasoning is what matters.
For a formative approach to work well, the classroom must provide a very supportive environment in which students are willing to voice their thinking. Reasoning must be stressed as more important than the answer. In our experience, students frequently give the right answer to the wrong problem. Their response is actually correct to a misinterpreted problem. If their misinterpretation is never revealed, being told that their answer is wrong only serves to confuse them.
It is certainly true that a formative approach takes more time and requires some reduction in topic coverage. Once instructors and students have acquired a little experience, however, one finds that the loss of topic coverage is not very great. Students become more adept at learning and do not need every topic and nuance mentioned in class. Also, if instructors use the information obtained in students’ responses, they find that they can spend less time on topics that are well understood.
Purpose of the Items
The intent of these items is to reveal student thinking, not to assess their knowledge. Many of the answers to the items have been included because they are common but incorrect choices. Some instructors are uncomfortable with this and would rather have all of the incorrect responses be obviously wrong. Further, they never use “none of the above” as the correct choice. They think these choices seduce students into making mistakes, and want students to be reinforced by finding the correct answer among the choices. This view may be appropriate for summative assessments, but it is completely incompatible with formative assessment.
Some of the items are straightforward and are used for one of two purposes. They can be used to explore a new topic and, if the majority of students respond correctly, it is possible to spend only a little time on this topic. This helps alleviate the pressure to complete a desired syllabus. Alternatively, these simple items can be used to confirm students after confusions have been resolved.
Promoting Discussion
The approach that we have found most effective is to allow students to discuss an item in small groups before providing their answer. This enables students both to test their reasoning on others and to rehearse their expression of their thinking. Students are much more willing to speak up in class if they have practiced their comments first.
After all the responses are tabulated (by a classroom response system), the distribution should be presented to the class as a whole. Knowing how everyone else in the class responded is one of the most powerful aspects of this approach. Students realize that they are not alone in their thinking, and that implicit assumptions play a significant role in problem solving.
A good technique is to ask for a volunteer to explain the answer he or she has given. It is important not to reveal by word or action whether the answer being defended is or is not correct. The important issue is whether or not the answer is logically defensible given the student’s interpretation of the question. After each of the submitted answers have been explained by someone, it is often useful to open the discussion with questions such as, “Has anyone heard anything that has caused them to change their mind?” Students are often more adept at recognizing flaws in another’s reasoning than in their own.
Interpreting Responses
In a formative assessment approach, it is the wrong answers that contain all the value. Each wrong response represents some flaw in a student’s thinking or in the communication process itself. It is essential that each wrong answer be analyzed to determine if it arises from a mathematical, conceptual, logical or communication error. For beginners, this is the time-consuming part of the approach. After a while, however, students will become quite facile at identifying the nature of the error.
Conceptual and logical errors should have the greatest impact on subsequent instruction. It is generally a good idea to tell students what is good about their thinking rather than what is bad. For example, rather than tell them they are wrong, identify the circumstances for which their answer would be correct. If they made some implicit assumption, then try to make them aware of the assumption.
Cognitive Load
Upon cursory inspection, many of the items in the library may appear duplicate. Every item is in fact distinct, but often a particular scenario is used many times. For beginning learners there is a lot of cognitive load associated with the context of a question. This load is reduced if the circumstance is familiar. Similarity of circumstance also permits more subtle conceptual distinctions to be drawn. For example, the velocity of a block that slides down a frictionless incline can be found equally efficiently by kinematics or energy principles, but when friction is present the energy method is simpler. On the other hand, if the time to reach the bottom is desired, kinematics is the preferred method even when friction is present.
You can find all of the questions using a particular scenario by searching the library using a meaningful part of the description of the situation. Unfortunately, this static archive of the library is not searchable. However, questions about a common scenario that are meant to be used together are usually arranged sequentially in the library (by item number).