Knowledge-lean Approaches to Statistical Natural Language Processing

The goal of natural language processing is to enable computers to use human language as a communication medium accurately, robustly, and gracefully. It is clear that a massive amount of knowledge, linguistic and otherwise, is needed to achieve this goal. As a result, much recent research has focused on getting computers to automatically learn high-quality information about language, and about the world, directly from the statistics of unprocessed or minimally processed language samples alone. We are not particular; any regularities in such samples that enable us to predict, classify, or otherwise characterize the apparent complexity of language for computational use is fair game. As examples, I will focus on two lines of work. The first uses information-theoretic distributional clustering methods trained on large language samples to induce sophisticated probabilistic models of linguistic co-occurrences. The second, in contrast, uses very simple statistics --- and no model at all --- to learn how to generate English versions of computer-generated proofs, creating texts whose quality rivals that of hand-crafted systems.

Portions of this talk are based on joint work with Regina Barzilay and with Fernando Pereira.

Home : News : Jamboree : 2003