Abstracts

Keynote Address: Where are we now? Where should we go? [Slides]

Karen Sparck Jones

Summarising covers a range from text extraction to content condensation. Its essential features are picking important concepts from, and reducing, source text or information, to deliver summary information or text. General strategies for doing this are clearly preferable to application-specific ones. So far, we have found that statistically-based sentence extraction and concatenation does not produce effective summaries. But we have not yet found general methods of content analysis and condensation. We can only identify key source content and present it in summary with heavy domain and goal guidance. The most pressing need is to develop `sufficient to the day' techniques that do more than surface sentence extraction without depending, MUC-like, on prior specifications for sought content. These needed intermediate techniques include passage extraction and linking; deep phrase selection and ordering; entity identification and relating. Such strategies benefit from, or require, shallow text analysis and do or can exploit statistical data. They may be enhanced by modern display resources. They are applicable to individual source texts or to data sets as wholes. Most importantly, we can tackle this level of summarising because current robust parsing technology may succeed, given source redundancy, in getting enough of value from sources to help users, and because current text production methods can deliver usable summary texts. We should push this line hard, seeking to minimise application-specific domain knowledge, to take advantage of discourse structure, and to address summary function for the user.

Salience-based Content Characterization of Text Documents

Branimir Boguraev and Christopher Kennedy

Traditionally, the document summarisation task has been tackled either as a natural language processing problem, with an instantiated meaning template being rendered into a coherent prose, or as a passage extraction problem, where certain fragments (typically sentences) of the source document are deemed to be highly representative of its content, and thus delivered as meaningful ``approximations'' of it. Balancing the conflicting requirements of depth and accuracy of a summary, on the one hand, and document and domain independence, on the other, has proven a very hard problem. This paper describes a novel approach to content characterisation of text documents. It is domain- and genre-independent, by virtue of not requiring an in-depth analysis of the full meaning. At the same time, it remains closer to the core meaning by choosing a different granularity of its representations (phrasal expressions rather than sentences or paragraphs), by exploiting a notion of discourse contiguity and coherence for the purposes of uniform coverage and context maintenance, and by utilising a strong linguistic notion of salience, as a more appropriate and representative measure of a document's ``aboutness''.

Using Lexical Chains for Text Summarization

Regina Barzilay and Michael Elhadad

We investigate one technique to produce a summary of an original text without requiring its full semantic interpretation, but instead relying on a model of topic progression in the text derived from lexical chains. We present a new algorithm to compute lexical chains in a text, merging several robust knowledge sources: the WordNet thesaurus, a part-of-speech tagger and shallow parser for the identification of nominal groups, and a segmentation algorithm derived from [Hearst 91]. Summarization proceeds in three steps: the original text is first segmented, lexical chains are constructed, strong chains are identified and significant sentences are extracted from the text. The paper provides empirical results on the identification of strong chains and the extraction of significant sentences.

Automated Text Summarization in SUMMARIST

Eduard Hovy and Chin Yew Lin

How to Appreciate the Quality of Automatic Text Summarization [Slides(RTF)]

Jean-Luc Minel, Sylvaine Nugier, and Gérald Piat

For the SERAPHIN project, we set up two assessment protocols in order to be able to more accurately assess the quality of abstracts - the FAN protocol and the MLUCE protocol, for which we provide the results. The FAN protocol assesses the legibility of an abstract, independently from the source text. The MLUCE protocol is designed to allow users of automatic abstracts to assess their quality. These protocols were applied to a corpus of 27 texts which varied in length from between three and twelve pages. These texts were randomly chosen from EDF archives. They include both scientific and general press articles, extracts from books, and internal EDF notes. The results of the FAN protocol demonstrate the difficulty of using surface linguistic indicators to assess the quality of an abstract; the results of the MLUCE protocol illustrate the importance of user expectations.

A Proposal for Task-Based Evaluation of Text Summarization Systems

Thérèse F. Hand

Evaluation is a key part of any research and development effort, but the goals and focus of evaluations are often narrow in scope, addressing a specific algorithm or technique, or analyzing a single result. All of the evaluation work done to date on text summarization systems has been by the developers of individual systems, usually to study and improve sentence selection criteria. Under TIPSTER III, DARPA is sponsoring a task-based evaluation of multiple text summarization systems. This focus of this evaluation will be on user needs, and the feasibility of applying summarization technology to a variety of tasks.

Automatic Text Summarization by Paragraph Extraction

Mandar Mitra, Amit Singhal, and Chris Buckley

Over the years, the amount of information available electronically has grown manifold. There is an increasing demand for automatic methods for text summarization. Domain-independent techniques for automatic summarization by paragraph extraction have been proposed in \cite{science,ht96}. In this study, we attempt to evaluate these methods by comparing the automatically generated extracts to ones generated by humans. In view of the fact that extracts generated by two humans for the same article are surprisingly dissimilar, the performance of the automatic methods is satisfactory. Even though this observation calls into question the feasibility of producing perfect summaries by extraction, given the unavailability of other effective domain-independent summarization tools, we believe that this is a reasonable, though imperfect, alternative.

Goal Directed Approach for Text Summarization

Ryo Ochitani, Yoshio Nakao, and Fumihito Nishino

The approach described in this report is intended to be a basic architecture to extract a set of concise sentences that are indicated or predicted by goals and contexts. To evaluate a sentence, the sentence selection algorithm simply measures the informativeness of each sentence by comparing with the determined goals, and the algorithm extracts a set of the highest scored sentences by repeat application of this comparison.

This approach is applied in the summary of Japanese news articles. The summaries consist of about 30% of the original text. On average, this method extracts 50% less text than the simple title-keyword method.

Statistical Methods for Retrieving Most Significant Paragraphs in Newspaper Articles

José Abracos and Gabriel Pereira Lopes

Retrieving a most significant paragraph in a newspaper article can act as a kind of summarization. It can give the human reader some hints on the contents of the article and help him to decide whether it deserves a full reading or not. It may also act as a filter for a robust natural language understanding system, to extract relevant information from that paragraph in order to enable conceptual information retrieval.

Taking a newspaper article and a base corpus, word co-occurrences with higher resolving power are identified. These co-occurrences are used to establish links between the paragraphs of the article. The paragraph which presents the larger number of links to other paragraphs is considered a most significant one.

Though designed and tested for the Portuguese language, the statistical nature of our proposal should ensure its portability to other languages.

Sentence extraction as a classification task [Slides] [Paper-Postscript] [Paper-HTML]

Simone H. Teufel and Marc Moens

A useful first step in document summarisation is the selection of a small number of `meaningful' sentences from a larger text. Kupiec et al. (1995) describe this as a classification task: on the basis of a corpus of technical papers with summaries written by professional abstractors, their system identifies those sentences in the text which also occur in the summary, and then acquires a model of the `abstract-worthiness' of a sentence as a combination of a limited number of properties of that sentence.

We report on a replication of this experiment with different data: summaries for our documents were not written by professional abstractors, but by the authors themselves. This produced fewer alignable sentences to train on. We use alternative `meaningful' sentences (selected by a human judge) as training and evaluation material, because this has advantages for the subsequent automatic generation of more flexible abstracts. We quantitatively compare the two different strategies for training and evaluation (viz. alignment vs. human judgement); we also discuss qualitative differences and consequences for the generation of abstracts.

A Scalable Summarization System Using Robust NLP

Chinatsu Aone, Mary Ellen Okurowski, James Gorlinsky, Bjornar Larsen

We describe a scalable summarization system which takes advantage of robust NLP technology such as corpus-based statistical NLP techniques, information extraction and readily available on-line resources. The system attempts to compensate for the bottlenecks of traditional frequency-based, knowledge-based or discourse-based summarization approaches by utilizing features derived by these robust techniques. Preliminary evaluation results are reported, and the multi-dimensional summary viewer is described.

COSY-MATS: An Intelligent and Scalable Summarization Shell

Maria Aretoulaki

In this paper, an architecture is presented for robust and portable summarisation, COSY-MATS. COSY-MATS can avoid the superficiality and domain-dependence of IE approaches by means of high-level (pragmatic and rhetorical) content selection features. It can also obviate the text type-dependence and cumbersome computation involved in NLU-based summarisation systems, because surface criteria are additionally used in the content selection process, as are identified mappings between those and the high-level features. In this way, COSY-MATS should retain its generic and scalable character, while also permitting intelligent application-specific processing.

From Discourse Structures to Text Summaries

Daniel Marcu

We describe experiments that show that the concepts of rhetorical analysis and nuclearity can be used effectively for determining the most important units in a text. We show how these concepts can be implemented and we discuss results that we obtained with a discourse-based summarization program.

SimSum: Simulation of summarizing

Brigitte Endres-Niggemeyer

SimSum (Simulation of Summarizing) simulates 20 real-world working steps of expert summarizers. It presents an empirically founded cognitive model of summarizing that operationalizes the discourse processing model developed by van Dijk and Kintsch (1983). The observed strategies of expert summarizers have given rise to cooperating object-oriented agents communicating through dedicated blackboards. Each agent is implemented as a CLOS object with an assigned actor at the multimedia user interface. The interface is realized with Macromedia Director. Communication between CLOS and Macromedia Director is mediated by Apple Events.

A Formal Model of Text Summarization Based on Condensation Operators of a Terminological Logic

Ulrich Reimer and Udo Hahn

We present an approach to text summarization that is entirely rooted in the formal description of a classification-based model of terminological knowledge representation and reasoning. Text summarization is considered an operator-based transformation process by which knowledge representation structures, as generated by the text understander, are mapped to conceptually condensed representation structures forming a text summary at the representation level. The framework we propose offers a variety of subtle parameters on which scalable text summarization can be based.