Automated Essay Scoring – An Application of the Text
Classification literature
Lawrence M. Rudner
University
of Maryland
ERIC
UMIACS Computational Linguistics Colloquium
October
29, 2003, 11:00am, AVW Room 2120
Two Bayesian models for text classification were extended and applied to
student produced essays. Both models were calibrated using 462 essays with two score points. The calibrated systems were
applied to 80 new, pre-scored essays with 40 essays in each score group.
Manipulated variables included the two models; the use of words, phrases and
arguments; two approaches to trimming; stemming; and the use of stopwords. While the text classification literature
suggests the need to calibrate on thousands of cases per score group, accuracy
of over 80% was achieved with the sparse dataset used in this study. A copy of
the paper Dr. Rudner will be presenting is at http://www.bc.edu/research/intasc/jtla/journal/pdf/v1n2_jtla.pdf.
About the speaker:
Lawrence M. Runder is the Director of the ERIC
Clearinghouse on Assessment and Evaluation at the University
of Maryland, College
Park. He holds
a Ph.D. in Psychology and Evaluation from Catholic
University, and his present research
interests include measurement decision theory and automated essay scoring. More information is available at http://ericae.net/~rudner.
For the colloquium series schedule, see the UMD Computational
Linguistics Colloquium Series web page at
http://umiacs.umd.edu/~resnik/cl_colloquium/. If you are interested in meeting
with the speaker, please contact Doug
Oard (oard@umiacs.umd.edu).