Instructions for Annotating Discourse

Abstract

This guide contains instructions for segmenting discourse that are meant to be self-contained. Although based on a particular theory of discourse structure, they do not make reference in any way to that theory. As a result, they can be used by "naive" subjects, i.e., segmenters who have not studied discourse theory or discourse processing methods. Indeed, they were designed explicitly for use by such subjects. An earlier version of the instructions was used by students in a discourse class before exposure to the theory on which the instructions are based. We augmented and modified the instructions based on this experience and are currently using them with naive subjects. We are disseminating the instructions so that others may use and experiment with them. We made two assumptions about the annotation environment in constructing this guide. First, we presume that segmenters will use an on-line marking tool to do their segmentation; we use Flammia's N. b. (Nota Bene) (Flammia and Zue, 1995). Second, we assume that the input text they are given to segment will be divided in advance by the experimenters into phrases; we use the intermediate phrase (Pierrehumbert, 1980; Beckman and Pierrehumbert, 1986) to determine these phrases in our studies (Hirschberg and Grosz, 1992; Grosz and Hirschberg, 1992, inter-alia). In addition, it is important to be aware of two places in which the guide makes reference to experimental options. First, the introduction to Section 2 allows for questioning the experimenter about the examples in the guide. Second, Section 4 introduces a number of special relationships between segments, not all of which may be relevant to the discourses or segmentation needs of other experiments. Thus, before employing this guide, experimenters should choose a marking tool, determine minimal units for annotation and divide up the discourses to be annotated accordingly, decide the extent to which they will allow questioning, and identify those sections of the guide that are relevant. We also recommend that segmenters be given adequate practice on training examples before they are asked to analyze discourses for your studies. The cooking instructions that are used for most of the examples in this guide have been adapted from a dialogue recorded at AT&T Bell Laboratories, Murray Hill, New Jersey, on 4 February 1987. We would like to acknowledge the researchers involved in making the original recording: Joan Bachenko, Bruce Ballard, Ron Brachman, Eileen Fitzpatrick, Mark Jones, Diane Litman, and Guy Story; special thanks to Eileen Fitzpatrick for making the transcripts available to us. The remaining examples, as well as the cooking instruction adaptations, were constructed by the authors.