TestBed

From TeacherWiki

The ASSISTments Research TestBed is the term that refers to the organization that combines the ASSISTments Platform, the ASSISTment School Collaborative, and ASSISTments Researcher Collaborative. The mission of the TestBed is two fold:

1. To do a social good by giving ASSISTments to teachers and students for free.
2. To do a lasting good by producing high quality research.

Neil Heffernan's collaborator, James Pellegrino (Distinguished Professor of Cognitive Psychology and Education, and Co-Director of the Center for the Study of Learning, Instruction and Teacher Development, University of Illinois at Chicago) had this to say:

“The combination of WPI’s ASSISTments system and their collaborative relationships with numerous schools and teachers provides an incredible testbed for designing and evaluating revisions to the CMP curriculum and instructional materials. There is no place else in the country where such a capability exists that fits so well with the goals of the National Center R&D plan.”

Fact Sheet about ASSISTments to download a copy. See how ASSISTments came to be, read the History of ASSISTments

Goals of the ASSISTments TestBed

1. Optimizing student learning: (What works, for whom, and when.)
   1. Which web based resources are most effective for helping students learn?
   2. How best to space out practice opportunities. (See Recent Math Center Grant)
   3. How to make math textbooks better.
2. Assessment:
   1. How to predict students' state test scores.
   2. How to predict students' knowledge using students' performance metrics.
   3. How to detect when students are not engaged and what to do about it.
   4. How to build tools to make intelligent tutoring cheaper and more widely used.
   5. How do teachers actually use technology in actual classrooms, and what are the ways they do so?

The TestBed is a Combination of a Tool, Schools, and Researchers

1. The ASSISTments Platform: It is the domain-general web-based system that allows teachers to write individual ASSISTments (composed of questions and associated hints, solutions, web-based videos etc.). It supports the creation, deployment, and reporting on randomized controlled experiments.
2. The ASSISTments School Collaborative: This collaborative started with Worcester Public Schools, and has grown to include about 100 teachers from 25 districts. This collaborative supports research studies that uses ASSISTments (e.g., Rittle Johnson) but the overwhelming majority have used ASSISTments to collect the data.
3. The ASSISTments Researcher Collaborative: This set of researchers is a group who use ASSISTments as part of their work. Fourteen PhD students have been involved with the ASSISTments research from eight universities and have produced over 30 peer-reviewed papers.

While Neil Heffernan was involved in getting almost all of them into using ASSISTments for research, he is a second author with a PhD on 20 papers, indicating he was not always the lead. Neil is always thrilled to get other researchers to see the power in the TestBed. He takes pride in building a TestBed that will produce research even if he is not the one leading the research and getting his name on the authors list. It's like building a giant scientific instruction that others can use to create good science.

Neil is pleased now to see new publications on which he has no authorship coming out now. He counts 10 such papers, of which seven have a WPI professor leading and three with no WPI faculty involved. Neil has given his data away freely to five other researchers for their own data mining research projects as well as to a handful of researchers outside WPI, and hopes to see this grow.

The ASSISTments TestBed Viewed as a "Collaboratory"

In the book Scientific Collaboration on the Internet, there is a chapter that defines a "Collaboratory". It is a term that applies in multiple ways to the ASSISTments TestBed.

• "A Collaboratory is an organizational entity that spans distance, supports rich and recurring human interaction oriented to a common research area, and fosters contact between researchers who are both known and unknown to each other, and provides access to data sources, artifacts, and tools required to accomplish research tasks."

They identified seven types of Collaboratories, and the ASSISTments TestBed shares aspects with all seven types.

1. Distributed Research Center - ASSISTments TestBed has over a dozen universities involved.
2. Shared Instruments - The ASSISTments Platform is a shared instrument, similar to the Hubble Space telescope or an electron microscope, except that multiple researchers can use it at the same time.
3. Community Data Systems - The repository of student data is shared with others.
4. Open Community Contribution Systems - Wikipedia is an example of one of these. ASSISTments is similar in that any teachers can make their own questions and tutoring. We avoid the problem of Wikipedia, in that no one can change a question a teacher owns, but they can make a copy. ASSISTments is also similar Youtube.
5. Virtual Communities of Practice – We have our own Google Group to help teams of teachers work together.
6. Virtual Learning Communities - ASSISTments can be used as a part of a virtual high school.
7. Community Infrastructure Projects - Community Infrastructure Projects seek to develop infrastructure to further work in a particular domain. This is an opportunity for teachers to work together to create content. Here the teachers are not necessarily acting as scientists, but instead focus on a project that helps their students while simultaneously helping others.

In Chapter 19 of their book, Matthew Bietz, Marsha Naidoo, and Gary Olson, describe a partnership between AIDS researchers in the United States and South Africa. Both sides stood to benefit from this cooperation: the U.S. researchers needed access to the untreated subject population, and the South Africans wanted to improve their infrastructure as well as make progress finding a cure for AIDS. Neil makes an analogy: ASSISTments is looking to find, not drugs to use for AIDS, but interventions that work to help schools teach better. Working along with the states, schools, and the teachers, we want to find out what works best as well.

This page was created by C. Heffernan, 2011.