<?tei:SOF p2aila.odd ?>
<!-- Revisions:  -->
<!-- 9 jan 94: LB: analysis is now called interpretation -->
<!-- 93-07-05 : DTL : renamed AIla, moved from CC per Eynsham -->
<!-- decision.  Redone to reconcile with rest of AI chapter.  -->
<!-- Some details and examples deleted.                       -->
<!-- 93-03-11 : MSM : delete indented version of CLAWS output -->
<!-- 93-03-10 : MSM : correct constit. analysis               -->
<!-- 93-03-10 : MSM : bowdlerize bibl citation                -->
<!-- 93-03-10 : MSM : revise for publication:                 -->
<!--                  change stenstrom:  oelig or oslash ?    -->
<!-- 93-03-08 : MSM : put sgml comments around ellipses       -->
<!-- 93-03-08 : MSM : put citn DATE into IMPRINT              -->
<!-- Date: Sat, 06 Mar 1993 22:01:14 +0000                    -->
<!-- From: Lou Burnard < lou@vax.ox.ac.uk >                   -->
<!-- To: U35395@UICVM.UIC.EDU                                 -->
<!-- CC: lou@vax.ox.ac.uk                                     -->
<!-- Subject: CCan.odd (6 of 10)                              -->
 
<div2 id=AILA><head>Linguistic Annotation</head>
 
<p>By <term>linguistic annotation</term> we mean here any annotation
determined by an analysis of linguistic features of the text, excluding
as borderline cases both the formal structural properties of the text
(e.g. its division into chapters or paragraphs) and descriptive
information about its context (the circumstances of its production, its
genre or medium).  The structural properties of any TEI-conformant text
should be represented using the structural elements discussed elsewhere
in this chapter and in chapters <xref target=CO>, <xref target=DS>, and
the various chapters of Part III (on base tag sets).  The contextual
properties of a TEI text are fully documented in the TEI Header, which
is discussed in chapter <xref target=HD>, and in section <xref
target=CCAH>.
 
<p>Other forms of linguistic annotation may be applied at a number of
levels in a text.  A code (such as a word-class or part-of-speech
code) may be associated with each word or token, or with groups of such
tokens, which may be continuous, discontinuous or nested.  A code may
also be associated with relationships (such as cohesion) perceived as
existing between distinct parts of a text.  The codes themselves may
stand for discrete non-decomposable categories, or they may represent
highly articulated bundles of textual features.  Their function may be
to place the annotated part of the text somewhere within a narrowly
linguistic or discoursal domain of analysis, or within a more general
semantic field, or any combination drawn from these and other domains.
 
<p>The manner by which such annotations are generated and attached to
the text may be entirely automatic, entirely manual or a mixture.  The
ease and accuracy with which analysis may be automated may vary with the
level at which the annotation is attached.  The method employed should
be documented in the <gi>interpretation</gi> element within the encoding
description of the TEI Header, as described in section <xref
target=HD53>.
<!-- To EDITORS from DTL:  Retain the next sentence here, or  -->
<!-- move it to CC chapter?                                   -->
<!-- Seems all right here, as a reminder.  (msm)              -->
Where different parts of a language corpus have used
different annotation methods, the <att>decls</att>
attribute may be used to indicate the fact, as further
discussed in section <xref target=CCAS>.
 
<!-- < div3 id=CCAN2 >< head >An Extended Example< / head> -->
<p>As one example of such types of analysis, consider the following
sentence, taken from the Lancaster/IBM Treebank Project.<note>
 
See <citation> <author>G.  N. Leech and R. G. Garside,</author>
<title.p>Running a Grammar Factory,</title.p> in
<title>English Computer Corpora:
Selected Papers and Research Guide,</title>
ed. <editor>S. Johansson and A.-B. Stenstr&oslash;m</editor>
(Berlin: de Gruyter; New York:  Mouton, 1991),
pp. 15-32.
</citation>
 
This sentence and its analysis are reproduced by kind permission of the
University of Lancaster's Unit for Computer Research on the English
Language.
 
<!-- Stenstrom's name changed from oelig, on theory that      -->
<!-- he's Scandinavian not French.  Pls check.  -MSM          -->
 
<!-- she's Swedish. Please restore her o-slash (LB) -->
</note>
 
<!-- [N [G The_AT victim_NN1 's_$ G] friends_NN2 N] [V told_VVD  -->
<!-- [N police_NN2 N] [Fn that_CST [N Krueger_NP1 N] [V [V&   -->
<!-- drove_VVD [P into_II [N the_AT quarry_NN1 N]P]V&]        -->
<!-- and_CC [V+ never_RR surfaced_VVD V+]V]Fn]V] ._.          -->
 
<q rend=display>
The victim's friends told police that Kruger drove into the quarry
and never surfaced.
</q>
 
<p>Our discussion focuses on the way that this sentence might be
analysed using the Claws system developed at the University of
Lancaster, but exactly the same principles may be applied to a wide
variety of other systems.
 
<note>For the word-class tagging method used by Claws see
 
<citation>
<author>I. Marshall</author>,
<title.p>Choice of Grammatical Word Class without Global Syntactic
Analysis:  Tagging Words in the LOB Corpus,</title.p>
in
<title>Computers and the Humanities</title>
<detail>17 (1983): 139-50.</detail>.
</citation>
 
For an overview of the system see
 
<citation>
<author>R. G. Garside, G. N. Leech, and G. R. Sampson,</author>
<title>The Computational Analysis of English: a Corpus-Based
Approach</title>
(Oxford:  Oxford University Press, 1991).
</citation>
</note>
 
Output from the system consists of a segmented and tokenized version of
the text, in which word class codes have been associated with each
token.  For our example sentence, we might conveniently represent these
codes using entity references:
 
<note place=foot>We have replaced the Claws code <code>&dollar;</code>
for the <cited>'s</cited> morpheme by <code>GEN</code>, as in the tag
set used by the British National Corpus (see <xref target=FSIL>), and
the code <code>.</code> for the final full stop by
<code>PUN</code>.</note>
 
<eg><![ CDATA [
<s>The&AT victim&NN1;'s&GEN friends&NN2 told&VVD police&NN2
that&CST Krueger&NP1  drove&VVD into&II the&AT
quarry&NN1  and&CC  never&RR surfaced&VVD;.&PUN
</s>
]]>
</eg>
 
The names used for these entity references have some significance for
the human reader (<code>AT</code> for <term>article</term>,
<code>NN1</code> for <term>singular noun</term>, <code>NN2</code> for
<term>plural noun</term>, etc.), but their representation in the
output from an SGML system processing the document may be adjusted, by
varying the entity declarations, to suit the convenience of whatever
analytic software is to be used.  For example, if the SGML parser
operating on this sentence uses a set of entity declarations in the
following form, then the word class tags will simply disappear from
the output.
 
<eg><![ CDATA [
<!ENTITY AT "">
<!ENTITY NN1 "">
<!ENTITY GEN "">
<!-- ... -->
]]>
</eg>
 
Alternatively, suppose the entity set in use follows the following
pattern:
 
<eg><![ CDATA [
<!ENTITY AT  "[definite article]">
<!ENTITY NN1 "[singular noun]">
<!ENTITY GEN "[genitive suffix]">
<!-- ... -->
]]>
</eg>
 
Then the sample sentence will be processed by an SGML-aware processor as
if it began:
 
<eg><![ CDATA [
The[definite article] victim[singular noun]'s[genitive suffix] ...
]]>
</eg>
 
<p>It would be more useful if the replacement texts for each entity
were a code of some significance to a particular analysis program.  If
the codes are considered to be <term>atomic</term>, then one of the
mechanisms based on the <gi>interp</gi> element described in section
<xref target=AISP> is sufficient.  If the codes are considered to be
compositional (for example that <code>NN1</code> and <code>NN2</code>
have something in common, namely their <term>noun-ness</term>, which
they do not share with, say, <code>VVD</code>), then this
compositionality may be most clearly expressed using a mechanism based
on the <gi>fs</gi> element defined in chapter <xref target=FS>.  For a
detailed example, see <xref target=FSIL>.
 
<p>One such replacement for the word-class entity references above
is a set of empty <gi>ptr</gi> elements bearing <att>target</att>
attributes as described in section <xref target=COXR>.  The required
entity definitions would look as follows.
 
<eg><![ CDATA [
<!ENTITY   AT   "<ptr target=AT>" >
<!ENTITY   NN1  "<ptr target=NN1>">
<!ENTITY   GEN  "<ptr target=GEN>">
<!-- ... -->
]]>
</eg>
 
Then the text would be expanded to read:
 
<eg><![ CDATA [
<s>The<ptr target=AT> victim<ptr target=NN1>'s<ptr target=GEN>
friends<ptr target=NN2> told<ptr target=VVD> police
<ptr target=NN2> that<ptr target=CST> Krueger<ptr target=NP1>
drove<ptr target=VVD> into<ptr target=II> the<ptr target=AT>
quarry<ptr target=NN1 and<ptr target=CC> never<ptr target=RR>
surfaced<ptr target=VVD>.<ptr target=PUN>
</s>
]]>
</eg>
 
<p>The <gi>ptr</gi> elements are designed to point to elements with
unique identifiers.  But we have yet to specify what those elements are.
Suppose we say that they are <gi>interp</gi> elements whose values are the
same as their identifiers.  That is, we provide an <gi>interpGrp</gi>
element as follows:
 
<eg><![ CDATA [
<interpGrp type='word classes'>
  <interp id=AT  value=AT>
  <interp id=NN1 value=NN1>
  <interp id=GEN value=GEN>
  <!-- ... -->
</interpGrp>
]]>
</eg>
 
<p>Although common practice, this (or any similar) method of relating
text to interpretation is seriously flawed.  The interpretations are
related not to text elements, but to points in the text, namely those
that are occupied by the <gi>ptr</gi> elements.  In order to relate the
interpretation to the appropriate text units, a uniform convention needs
to be applied; for example, that an interpretation relates to all the
text material preceding the <gi>ptr</gi> element that points to it up to
the immediately preceding <gi>ptr</gi>, or up to the <gi>s</gi> that
delimits the S-unit containing that <gi>ptr</gi> element, whichever is
nearer.  While this convention works with texts that are marked up
solely with <gi>ptr</gi> elements that point to interpretation elements,
it does not work with texts with additional markup, for example
<gi>ptr</gi> elements that are used for some other purpose.  In addition,
the convention fails for any markup in which interpretations are
intended to be associated with nested text elements.
 
<p>None of these difficulties arise if the text is fully segmented, using
the linguistic segment elements described in section <xref target=AILC>,
and the <att>ana</att> attribute to point to the interpretations
that are associated with each such segment, as follows:
 
<eg><![ CDATA [
<s type=sentence>
<w ana=AT >The</w>
<w ana=NN1>victim</w>
<m ana=GEN>'s</m>
<w ana=NN2>friends</w>
<w ana=VVD>told</w>
<w ana=NN2>police</w>
<w ana=CST>that</w>
<w ana=NP1>Krueger</w>
<w ana=VVD>drove</w>
<w ana=II >into</w>
<w ana=AT >the</w>
<w ana=NN1>quarry</w>
<w ana=CC >and</w>
<w ana=RR >never</w>
<w ana=VVD>surfaced</w>
<c ana=PUN>.</c>
</s>
]]>
</eg>
 
<p>Analysis into phrase and clause elements can be superimposed on the
word and morpheme tagging in the preceding illustration. For example,
Claws provides the following constituent analysis of the sample sentence
(the word class codes have been deleted):
 
<q rend=display>
[N [G The victim's G] friends N] [V told [N police N] [Fn that [N
Krueger N] [V [V& drove [P into [N the quarry N]P]V&]
and [V+ never surfaced V+]V]Fn]V]
</q>
 
<p>Treating the labels on the brackets as phrase or clause
interpretations, this analysis of the structure of the example sentence
can be combined with the word class analysis and represented as follows
(the symbol <code>V&</code> representing the first part of a coordinate
phrase, has been replaced by <code>V1</code>, and <code>V+</code>, representing the
second part, has been replaced by <code>V2</code>).
 
<eg><![ CDATA [
<s type=sentence>
<phr ana=N>
  <phr ana=G>
    <w ana=AT>The</w>
    <w ana=NN1>victim</w>
    <m ana=GEN>'s</m>
    </phr>
  <w ana=NN2>friends</w>
  </phr>
<phr ana=V>
  <w ana=VVD>told</w>
  <phr ana=N><w ana=NN2>police</w></phr>
  <cl ana=Fn>
    <w ana=CST>that</w>
    <phr ana=N><w ana=NP1>Krueger</w></phr>
    <phr ana=V>
      <phr ana=V1>
        <w ana=VVD>drove</w>
        <phr ana=P>
          <w ana=II>into</w>
          <phr ana=N>
            <w ana=AT>the</w>
            <w ana=NN1>quarry</w>
            </phr>
          </phr>
        </phr>
      <w ana=CC>and</w>
      <phr ana=V2>
        <w ana=RR>never</w>
        <w ana=VVD>surfaced</w>
        </phr>
      </phr>
    </cl>
  </phr>
<c ana=PUN>.</c>
</s>
]]>
</eg>
 
<p>A representation using the <gi>linkGrp</gi> element can be obtained by
supplying each linguistic segment with its own <att>id</att> attribute,
removing its <att>ana</att> attribute, and putting each
segment-interpretation pair into a <gi>link</gi> element inside the
<gi>linkGrp</gi> element.
 
<!-- < div3 id=CCAN3>< head>Non-nesting Structures< / head> -->
<p>Each linguistic segment so far discussed has been well-behaved with
respect to the basic document hierarchy, having only a single parent.
Moreover, the segmentation has been complete, in that each part of the
text is accounted for by some segment at each level of analysis, without
discontinuities or overlap.  This state of affairs does not of
course apply in all types of analysis, and these Guidelines provide a
number of mechanisms to support the representation of discontinuities or
multiple analyses.  A brief overview of these facilities is provided in
chapter <xref target=NH>; also see <xref target=SA>.  These mechanisms
all depend to a greater or lesser degree on the ability to associate a
unique identifier with any element in a TEI-conformant text, and then to
specify that identifier as the target of a pointing element of some kind.
 
<!-- < div3 id=CCAN4>< head>Further Examples< / head> -->
<p>The mechanisms proposed in this chapter may also be used to encode
analyses of an entirely different kind, for example discourse function.
Here is an application of the span technique to record details of a sales
transaction in a spoken text.
 
<eg>
<![ CDATA [
<u who=P1 id=U1>Can I have ten oranges and a kilo of 
    bananas please?</u>
<u who=P2 id=U2>Yes, anything else?</u>
<u who=P1 id=U3>No thanks.</u>
<u who=P2 id=U4>That'll be dollar forty.</u>
<u who=P1 id=U5>Two dollars</u>
<u who=P1 id=U6>Sixty, eighty, two dollars. Thank you.</u>
 
<spanGrp type=transactions>
  <span from=U1 value='sale request'>
  <span from=U2 to=U3 value='sale compliance'>
  <span from=U4 value='sale'>
  <span from=U5 value='purchase'>
  <span from=U6 value='purchase closure'>
  </spanGrp>
]]>
</eg><!-- R. Hasan, Language, context, and text... OUP1989, p 59 -->
 
For further discussion of the <gi>u</gi> (utterance) element and other
elements recommended for transcriptions of spoken language, see chapter
<xref target=TS>.
<?tei:EOF p2aila.odd ?>
