Publications
From TeacherWiki
Contents |
Getting Started Reading About ASSISTments
Professor Heffernan has over 30 publications related to ASSISTments, but most of them are aimed at other computer scientists and don't interest teachers. These papers are important scientific findings that hundreds of other researchers have cited. These studies are also important as without these scientific studies, we would have no federal funding. We get funded due to our reputation as scientists producing valuable findings. So to help practitioners I have assembled a few good starting points here.
Research Articles with Empirical Results Showing ASSISTments works
Homework Study
Here is an article that shows ASSISTmetns dramatically increases student knowledge when used for homework compared to traditional homework (feedback the next day).
- Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009) Comparison of Traditional Homework with Computer Supported Homework: Improving Learning from Homework Using Intelligent Tutoring Systems. Journal of Research on Technology in Education (JRTE). Published by the International Society For Technology in Education (ISTE). PDF
Study showing students that used Assistments learned more on a State Test
In this paper we report on the finding that schools that used ASSISTments learned more on the state test compared to a school in the same city that did not adopt ASSISTments.
- Koedinger, K., McLaughlin, E. & Heffernan, N. (in press) A Quasi-Experimental Evaluation of an On-line Formative Assessment and Tutoring System. Journal of Educational Computing Research. Baywood Publishing. PDF
Teachers and school administrators: Read these four articles to learn more about how it works
- While here is a book chapter prepared by Neil Heffernan for a book Chris Dede at Havard is editing. It is expected to be published in the fall of 2010 by Havard or Columbia's Teacher Press. Book Chapter
- Here is an article on our vision how ASSISTments could be used to kick start a Professional Learning Community. Article.
- Here is an article written for principals and superintendents. Article
- If you are looking for documentation on how to use ASSISTments you can read this user manual. User manual and Information for getting started.
Other Studies of interest
We do research into assessment of student and on the best way to improve student learning. We have over two dozen such articles, but the following are two good ones to start with.
Here is an article that is focused on the assessment results of interest to Educators.
- Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Intelligent Tutoring System that tutors as it assesses. The Journal of User Modeling and User-Adapted Interaction. Vol 19: p243-266. [1].
Here is a representative article concerning student learning.
- Razzaq, L. & Heffernan, N. (2009) To Tutor or Not to Tutor: That is the Question. In Dimitrova, Mizoguchi, du Boulay & Graesser (Eds.) Proceedings of the 2009 Artificial Intelligence in Education Conference. IOS Press. pp. 457-464.Honorable Mention for Best Student Paper [2].(Here is a text version of this page.)
Empirical work Listed by Year
Every one of these papers listed below were peer reviewed. Acceptance rates of most of these venues were in middle 30% (meaning that 70% were rejected).
For the most up to date listing of publications please visit Neil Heffernan's CV page
Recent/Accepted (2009-10)
Jozsef Patvarczki, Elke A. Rundensteiner, and Neil T. Heffernan, (2010). "Using Machine Learning Technique to Parallelize Databases: Where each query answered by a single node", New England DB Summit, MIT, Cambridge, 2010
Pardos, Z. A., Heffernan, N. T. (2009). Determining the Significance of Item Order In Randomized Problem Sets. In Proceedings of the 2nd International Conference on Educational Data Mining. Cordoba, Spain.
Pardos, Z. A., Heffernan, N. T. (2009). Detecting the Learning Value of Items In a Randomized Problem Set. In Proceedings of the 14th International Conference on Artificial Intelligence in Education. Brighton, UK. IOS Press.
Patvarczki, J., Politz, J., Heffernan, N.T. (2009). Scalability and Robustness in the Domain of Web Based Tutoring. In Proceedings of the 14th International Conference on Artificial Intelligence in Education, Workshop Proceedings Volume 4, Scalability Issues in AIED. Brighton, UK. , pp. 20-29.
Jozsef Patvarczki, Murali Mani, and Neil Heffernan, "Performance Driven Database Design for Scalable Web Applications", Advances in Databases and Information Systems, In J. Grundspenkis, T. Morzy & G. Vossen (Eds) Advances in Databases and Information Systems Springer-Verlag: Berlin. 2009, pp 43-58
Razzaq, L., Heffernan, N.T. (in press). To Tutor or Not to Tutor: That is the Question. To appear in the Proceedings of the Annual Conference on Artificial Intelligence in Education 2009.
Razzaq, L., Parvarczki, J., Almeida, S.F., Vartak, M., Feng, M., Heffernan, N.T. and Koedinger, K., (2009). "The ASSISTment builder: Supporting the Life-cycle of ITS Content Creation", IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems. 2(2), 157-166.
Shrestha, P., Wei, X., Maharjan, A., Razzaq, L., Heffernan, N.T., Heffernan, C. (2009). Are Worked Examples an Effective Feedback Mechanism During Problem Solving? Accepted to the Annual Meeting of the Cognitive Science Society, 2009.
2010 WPI GRAD Day
Elijah Forbes-Summers and Neil Heffernan, "Trying to Reduce Gaming Behavior by Students in Intelligent Tutoring Systems", 2010 WPI Grad Day, Worcester, 2010
Yue Gong, Joseph E. Beck, Neil T. Heffernan, Elijah Forbes-Summers, "The impact of gaming (?) on learning at the fine-grained level", 2010 WPI Grad Day, Worcester, 2010
Ryan S.J.d Baker, Adam B. Goldstein, and Neil T. Heffernan, "Detecting the Moment of Learning", 2010 WPI Grad Day, Worcester, 2010
Michael São Pedro, Janice Gobert, and Neil Heffernan, "Comparing Pedagogical Approaches for the Acquisition and Long-Term Robustness of the Control of Variables Strategy", 2010 WPI Grad Day, Worcester, 2010
Zachary A. Pardos and Neil T. Heffernan, "Navigating the parameter space of Bayesian Knowledge Tracing models: Visualizations of the convergence of the Expectation Maximization algorithm", 2010 WPI Grad Day, Worcester, 2010
Jozsef Patvarczki, Elke A. Rundensteiner, Craig E. Wills, and Neil T. Heffernan, "Using Machine Learning Technique to Parallelize Databases: Where each query answered by a single node", 2010 WPI Grad Day, Worcester, 2010
Zach Broderick, Christine O’Connor, Courtney Mulcahy, Cristina Heffernan, and Neil Heffernan, "Using an Intelligent Tutoring System to increase parent engagement in student learning", 2010 WPI Grad Day, Worcester, 2010
Zach Pardos, Matthew Dailey, Neil Heffernan, "Learning From Non-experimental Data", 2010 WPI Grad Day, Worcester, 2010
2009
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2009). Addressing the assessment challenge in an Online System that tutors as it assesses. To appear in User Modeling and User-Adapted Interaction: The Journal of Personalization Research (UMUAI journal).
Feng, M, Heffernan, N., Heffernan, C. & Mani, M. (2009). Using mixed-effects modeling to analyze different grain-sized skill models. Accepted by the IEEE Transactions on Learning Technologies Special Issue on Real-World Applications of Intelligent Tutoring Systems.
Mendicino, M., Razzaq, L. & Heffernan, N. T. (2009). Comparison of Traditional Homework with Computer Supported Homework: Improving Learning from Homework Using Intelligent Tutoring Systems. Journal of Research on Technology in Education (JRTE). Published by the International Society For Technology in Education (ISTE).
Militello, M., & Heffernan, N. (2009). Which one is "just right"? What educators should know about formative assessment systems. International Journal of Educational Leadership Preparation, 4(3), 1-8.
Patvarczki, J., Mani, M., and Heffernan, Neil. (2009). "Performance Driven Database Design for Scalable Web Applications", 2009 New England Database Day, MIT, Cambridge, 2009
2008
Almeida, S., Vartak, M., Heffernan, N. (2008). "Supporting the Generation of Isomorphic Tutoring Content," Proceedings of the Young Researchers Track ITS 2008, June 2008, Montreal
Ayers, E. and Junker, B.W. (2008). IRT Modeling of Tutor Performance To Predict End-of-year Exam Scores. Educational and Psychological Measurement, 68,972-987.
Guo Y., Beck, J.E., Heffernan, N.T. (2008). "Trying to Reduce Bottom-out hinting: Will telling student how many hints they have left help?", Intelligent Tutoring Systems 2008, Montreal, Canada
Feng, M., Beck, J,. Heffernan, N. & Koedinger, K. (2008). Can an Intelligent Tutoring System Predict Math Proficiency as Well as a Standardized Test? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.107-116. Montral 2008.
Feng, M., Heffernan, N., Beck, J, & Koedinger, K. (2008) Can we predict which groups of questions students will learn from? In Baker & Beck (Eds.). Proceedings of the 1st International Conference on Education Data Mining. pp.218-225. Montral 2008
Pardos, Z. A., Heffernan, N. T., Ruiz, C. & Beck, J. (2008). Effective Skill Assessment Using Expectation Maximization in a Multi Network Temporal Bayesian Network. The Young Researchers Track at the 20th International Conference on Intelligent Tutoring Systems. Montreal, Canada. pp. 31-40
Pardos, Z. A., Beck, J., Ruiz, C. & Heffernan, N. T. (2008). The Composition Effect: Conjunctive or Compensatory? An Analysis of Multi-Skill Math Questions in ITS. Proceedings of the First International Conference on Educational Data Mining. Montreal, Canada. pp. 147-156
Patvarczki, J., Almeida, S.F., Beck, J.E., Heffernan, N.T. (2008). "Lessons Learned from Scaling Up a Web-Based Intelligent Tutoring System", Intelligent Tutoring Systems 2008, Montreal, Canada, pp. 766-770
Razzaq, L., Mendicino, M. & Heffernan, N. (2008). Comparing classroom problem-solving with no feedback to web-based homework assistance. In Woolf, Aimeur, Nkambou and Lajoie (Eds.) Proceeding of the 9th International Conference on Intelligent Tutoring Systems. pp. 426 - 437. Springer-Verlag: Berlin.
Razzaq, L., Heffernan, N.T. (2008). Towards Designing a User-Adaptive Web-Based E-Learning System. In Mary Czerwinski, Arnold M. Lund, Desney S. Tan (Eds.): Extended Abstracts Proceedings of the 2008 Conference on Human Factors in Computing Systems, CHI 2008, pp. 3525-3530. Florence, Italy: ACM.
Razzaq, L., Patvarczki, J., Almeida, S., Vartak, M., Feng, M., Heffernan, N.T. & Koedinger, K.R. (2008). "The ASSISTment Builder: Supporting the Life Cycle of ITS Content Creation", WPI Tech report, WPI-CS-TR-08-06.
Vartak, M., Almeida, S., Heffernan, N. (2008). "Extending ITS Authoring Tools to be Culturally Aware,"Proceedings of the Workshop on Culturally Aware Tutoring Systems, June 2008, Montreal
Vartak, M., Almeida, S., Heffernan, N. (2008). "Cost-effective Content Creation with Variabilization," Technical Report Series, Computer Science Dept., W.P.I.
2007
Feng, M., Heffernan, N. T. (2007). Assessing Students. Performance Longitudinally: Item Difficulty Parameter vs. Skill Learning Tracking. Paper presented at the 2007 Annual meeting of National Council of Measurement on Educational (NCME'2007), Chicago.
Feng, M., Heffernan, N.T. (2007). Towards Live Informing and Automatic Analyzing of Student Learning: Reporting in the Assistment System. Special Issue of Journal of Interactive Learning Research (JILR) entitled "Usage Analysis in Learning Systems: Existing Approaches and Scientific Issues", 18(2).
Junker B.(2007). On-demand learning-embedded benchmark assessment using classroom-accessible technology. Proposal for a symposium at NCME Annual Conference 2007.
Pardos, Z., Feng, M. & Heffernan, N. T. & Heffernan-Lindquist, C. (2007) Analyzing fine-grained skill models using bayesian and mixed effect methods. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press.
Pardos, Z. A., Heffernan, N. T., Anderson, B. & Heffernan, C. (2007). The effect of model granularity on student performance prediction using Bayesian networks. The International User Modeling Conference 2007.
Razzaq, Feng, Heffernan, Koedinger, Nuzzo-Jones, Junker, Macasek, Rasmussen, Turner & Walonoski (2007). Blending Assessment and Instructional Assistance. In Nadia Nedjah, Luiza deMacedo Mourelle, Mario Neto Borges and Nival Nunesde Almeida (Eds). Intelligent Educational Machines within the Intelligent Systems Engineering Book Series . pp.23-49. (see http://www.isebis.eng.uerj.br/). Springer Berlin / Heidelberg.
Razzaq, L., Heffernan, N., Feng, M., Pardos Z. (2007). Developing Fine-Grained Transfer Models in the ASSISTment System. Journal of Technology, Instruction, Cognition, and Learning, Vol. 5. Number 3. Old City Publishing, Philadelphia, PA. pp. 289-304.
Razzaq, L., Heffernan, N. T., Lindeman, R. W. (2007) What level of tutor interaction is best? In Luckin & Koedinger (Eds) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press. pp. 222-229.
Wietz, R., Heffernan, N. T., Kodaganallur, V. & Rosenthal, D. (2007). The distribution of student errors across schools: An initial study. In Luckin & Koedinger (Eds.) Proceedings of the 13th Conference on Artificial Intelligence in Education. IOS Press.
2006
Anozie N., & Junker B. W. (2006). Predicting end-of-year accountability assessment scores from monthly student records in an online tutoring system. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 1-6. Technical Report WS-06-05.
Ayers E., & Junker B. W. (2006). Do skills combine additively to predict task difficulty in eighth grade mathematics? In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 14-20. Technical Report WS-06-05.
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006a). Addressing the Testing Challenge with a Web-Based E-Assessment System that Tutors as it Assesses. Proceedings of the Fifteenth International World Wide Web Conference. pp. 307-316. New York, NY: ACM Press. 2006. ISBN:1-59593-332-9. Nominated for "Best Student Paper".
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2006b). Predicting state test scores better with intelligent tutoring systems: developing metrics to measure assistance required. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 31-40. 2006.
Feng, M., Heffernan, N. T., Mani, M., & Heffernan, C. (2006). Using Mixed-Effects Modeling to Compare Different Grain-Sized Skill Models. In Beck, J., Aimeur, E., & Barnes, T. (Eds). Educational Data Mining: Papers from the AAAI Workshop. Menlo Park, CA: AAAI Press. pp. 57-66. Technical Report WS-06-05. ISBN 978-1-57735-287-7.
Feng, M., Heffernan, N.T. (2006). Informing Teachers Live about Student Learning: Reporting in the Assistment System. To be published in Technology, Instruction, Cognition, and Learning Journal Vol. 3. Old City Publishing, Philadelphia, PA. 2006.
Heffernan N.T., Turner T.E., Lourenco A.L.N., Macasek M.A., Nuzzo-Jones G., Koedinger K.R., The ASSISTment Builder: Towards an Analysis of Cost Effectiveness of ITS creation, Accepted by FLAIRS2006, Florida, USA (2006).
Junker, B. W. (2006). Using On-line Tutoring Records to Predict End-of-Year Exam Scores: Experience with the ASSISTments Project and MCAS 8th Grade Mathematics. To appear in Lissitz, R. W. (Ed.), Assessing and modeling cognitive development in school: intellectual growth and standard settings. Maple Grove, MN: JAM Press.
Kardian, K., Heffernan, N.T. (2006). Knowledge Engineering for Intelligent Tutoring Systems: Assessing Semi-automatic Skill Encoding Methods. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 735-737
Pardos, Z. A., Heffernan, N. T., Anderson, B., & Heffernan C. (2006). Using Fine-Grained Skill Models to Fit Student Performance with Bayesian Networks. Workshop in Educational Data Mining held at the 8th International Conference on Intelligent Tutoring Systems. Taiwan. 2006.
Rasmussen, K., Patvarczki J. & Heffernan, N.T. (2006) "Scaling up the Server-based ASSISTment ITS: Speed and Reliability", WPI Tech Report, WPI-CS-TR-06-12.
Razzaq, L., Heffernan, N.T. (2006). Scaffolding vs. hints in the Assistment System. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 635-644. 2006.
Walonoski, J., Heffernan, N.T. (2006). Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 382-391. 2006
Walonoski, J., Heffernan, N. T. (2006). Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems. In Ikeda, Ashley & Chan (Eds.). Proceedings of the 8th International Conference on Intelligent Tutoring Systems. Springer-Verlag: Berlin. pp. 722-724. 2006
2005
Feng, M., Heffernan, N.T. (2005). Informing Teachers Live about Student Learning: Reporting in the Assistment System. the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Usage Analysis in Learning Systems, 2005, Amsterdam.
Feng, M., Heffernan, N.T., & Koedinger, K.R. (2005). Looking for Sources of Error in Predicting Student's Knowledge. In Beck. J (Eds). Educational Data Mining: Papers from the 2005 AAAI Workshop. Menlo Park, California: AAAI Press. pp. 54-61 Technical Report WS-05-02. ISBN 978-1-57735-238-9
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 902-904. Amsterdam: ISO Press.
Nuzzo-Jones, G., Walonoski, J.A., Heffernan, N.T., Livak, T. (2005). The eXtensible Tutor Architecture: A New Foundation for ITS. In Proceedings of the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
Razzaq, L., Feng, M., Nuzzo-Jones, G., Heffernan, N.T., Koedinger, K. R., Junker, B., Ritter, S., Knight, A., Aniszczyk, C., Choksey, S., Livak, T., Mercado, E., Turner, T.E., Upalekar. R, Walonoski, J.A., Macasek. M.A., Rasmussen, K.P. (2005). The Assistment Project: Blending Assessment and Assisting. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 555-562. Amsterdam: ISO Press.
Rose, C., Donmez, P., Gweon, G., Knight, A., Junker, B., Cohen, W., Koedinger, K., Heffernan, N. (2005). Automatic and Semi-Automatic Skill Coding With a View Towards Supporting On-Line Assessment. the 12th International Conference on Artificial Intelligence in Education 2005, Amsterdam
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid Development Tool for ITS. In C.K. Looi, G. McCalla, B. Bredeweg, & J. Breuker (Eds.) Proceedings of the 12th International Conference on Artificial Intelligence In Education, 929-931. Amsterdam: ISO Press.
Turner, T.E., Macasek, M.A., Nuzzo-Jones, G., Heffernan, N..T, Koedinger, K. (2005). The Assistment Builder: A Rapid Development Tool for ITS. Poster in the 12th International Conference on Artificial Intelligence in Education 2005 Workshop on Adaptive Systems for Web-Based Education: Tools and Reusability, Amsterdam
