ARRS Grant

From TeacherWiki

Taking advantage of Cognitive Science Principles: Adding to a Computer-Based tutor an Automatic Reassessment and Relearning System (ARRS)

This grant is NSF# DRL-1109483. We expect $750,000 over three years starting July 1, 2011. To learn use are go here for the ARRS documentation.

• PI: Joseph Beck
• Co-PI: Neil Heffernan
• Worcester Polytechnic Institute

This Empirical Research (Contextual Research) proposal focuses on transferring known effects in the cognitive science literature, spacing and testing, and using computer support to make them practical for use by classroom mathematics teachers. A recent (2007) Practice Guide by the Institute for Education Sciences entitled “Organizing Instruction and Study to Improve Student Learning” reports “that cognitive science research supports the idea that teachers can improve student learning by making sure they come back to topics periodically to ensure students are retaining knowledge.” The report points out that this idea is supported by the well known “spacing effect” that shows students learn more if their practice opportunities are spaced out over time rather than massed together in a single practice session.

In addition to spacing, we will leverage the idea of “mastery learning,” where students practice until they reach some mastery threshold. Researchers have shown this to be effective, but it was not widely adopted until computers were employed to do the tedious bookkeeping. Similarly, the recommendation of re-quizzing students to ensure they retain knowledge is very difficult to fully implement without computer support. We will extend this idea by using concepts from the NSF-funded Pittsburgh Science of Learning Center. They argue that learning should be robust, by which they mean it should be retained for a long period of time, aid in future learning, and transfer to novel situations. This proposal focuses on the first two aspects of robust learning. In contrast, current computer based systems that implement mastery learning use a criterion that allows students to reach “mastery” without first having to demonstrate retention over time. We propose to see if we can improve student learning when the mastery criterion is changed to incorporate a more robust measure of learning. Practically, this means that after students have demonstrated short-term retention on a given day, they will have that information reassessed on a given schedule, which progressively lengthens the time between practice opportunities; for instance, one day later, followed by a week later, followed by 2 weeks after that, and one month after that. If students get the subsequent reassessment questions wrong, they will get support in re-mastering the topic in the form of intelligent tutoring support for an existing NSF funded tutor. We hypothesize that such an intervention will not only increase student retention, but will accelerate student future learning of related mathematics skills.

In this grant we propose to evaluate our intervention for robust learning. Along the way, we will answers some targeted research questions that will be contributions to the field even if our intervention is not an improvement over classic mastery learning. Main research question: Does this intervention produce robust learning?

1. What is the most effective spacing for learning? If some knowledge components are easier to forget, should they get different re-quizzing schedules?
2. Can we do a better job of tracking student knowledge if we model individual student forgetting rates? By doing so, can we use time more efficiently by customizing our re-quizzing schedule to the individual?
3. If we control for student knowledge, is this intervention a good use of time compared to current practice?
4. Will students who receive our intervention have better retention?
5. Will student who receive our intervention demonstrate accelerated future learning of similar skills?
6. Can teachers use the computer reports on who is not retaining information well to help these students?

This grant’s intellectual merit will result in being able to show how to leverage the cognitive science principles of spacing and testing effects to make a computer based system that leads to better student learning by adding a feature of automatic spacing and review. The broader impact of this work will be the thousands of students will benefit by their teacher being able to use this system, or another system based off of our results, that implements these features that contains the “desirable difficulties” that, while being effective, are currently impractical for teachers to implement without technological support