The past decade has witnessed a resurgence of attention to the role of statistical induction in human language learning. One line of such work argues that proper attention to statistical patterns overcomes traditional "poverty of the stimulus" arguments, and thereby obviates the need for innate grammatical knowledge. In this talk, I will survey the current state of this debate, considering what kinds of information has and has not been shown to be extractable by statistical techniques. I will focus attention on the error-driven training of artificial neural networks, specifically Elman's Simple Recurrent Networks (SRNs), as this constitutes a particularly flexible and powerful technique for statistical induction, and is the one that has achieved what appear to be the most impressive results to date. I will then report on a line of work (conducted in collaboration with Don Mathis and Bill Badecker) that explores the capacity of SRNs to extract grammatical generalizations. In one set of experiments, we explore the ability of these networks to learn the structuresensitive generalization involved in grammatical transformations (such as passive and question formation). In another we consider the task of learning constraints on referential dependencies in reflexive and pronominal anaphora. These tasks requires a more refined sensitivity to grammatical structure than those studied previously. We find that the statistical nature of the training yields the SRN to a solution that is successful when measured quantitatively on data of the sort on which it was trained, but which diverges in certain key respects from the target generalization. We also find variation in the network's ability to extend its grammatical generalizations to novel structures depending on the conditions of the training, but find no ability at all to generalize to novel lexical items.
Bob Frank is Professor in the Cognitive Science department at Johns Hopkins University. After receiving his PhD in Computer and Information Science from the University of Pennsylvania in 1992, he taught in the Linguistics department at the University of Delaware. Bob's departmental meanderings point to his interdisciplinary research interests, applying insights from mathematical formalization and computational modeling to problems in syntactic theory, language acquisition and processing.
This talk is part of the CLIP Colloquium Series, organized by Jimmy Lin (jimmylin -at- umd .dot. edu). For the complete schedule, please visit http://www.umiacs.umd.edu/research/CLIP/colloq/.