\documentstyle[12pt]{article}
\begin{document}

\hyphenation{Com-pu-ter-Pro-cess-able}

\begin{center}

{\small \bf Guidelines for Writing SGML DTD's and Instances of SGML
DTD's}

S.A. Mamrak, J. Barnes and M.J. Kaelbling\\

\today
\end{center}

\section{Introduction}

The Text Encoding Initiative (TEI) has decided to use SGML as the
basis for representing encoded documents.  SGML requires that Document
Type Definitions, DTD's, be specified for each type of document to be
encoded.  DTD's define a tag set, and the rules for ordering tags.
The rules are specified using an extended BNF notation.  Individual
documents, i.e., datasets, can then be marked up using these tag sets.
These individually marked up datasets are called instances of DTD's.

The responsibilities of the Syntax and Metalanguage Committee of
the TEI are (1) to determine if SGML is powerful enough to meet the
analysis needs of the TEI community, and (2) to ensure that SGML DTD's
and DTD instances defined in the project are amenable to processing by
computers.  The nature of analyses to be undertaken by the TEI
community is currently under independent evaluation by other TEI
Committees.  Their findings will lay the basis for determining the
sufficiency of SGML.  It is the purpose of this paper to make
recommendations that will ensure that the DTD's and their instances,
as they are defined within the TEI project, are amenable to processing
by computers.

\section{Processing SGML with Computers}

In this paper, guidelines will be suggested for defining SGML DTD's
and instances of DTD's.  As the grammar describing SGML has over 200
productions and many pages of explanation, elaboration and
clarification, it is inappropriate to present a detailed description
here.  Therefore, the interested reader is referred to the standard
itself for a detailed description of SGML.  The complete description,
however, is not necessary for an understanding of the guidelines given
below.

\subsection{The Use of Existing Software Tools}

Since SGML is a language intended for computer-based systems, it is
reasonable that it should follow established conventions within the
realm of computer science.  By following accepted methods of notation
and structural configuration, languages can be processed by existing
tools following well-understood theories and techniques, thus offering
considerable savings to development efforts.  These savings are
realized by automatically performing tedious, error-prone calculations
correctly and by checking and clarifying the formal descriptions of
the languages.

Tokenizing and parsing are of major concern to computer science.
Tokenizing is the process of recognizing significant strings in a
language and parsing is the process of recognizing the language
itself.  There are several common techniques of tokenizing and
parsing, each with known properties and limitations.  Some of these
techniques are so well established that programs exist that can
automatically generate tokenizers and parsers for conforming language
descriptions.  The use of these tokenizer and parser generators goes a
long way to ensuring the rapid production of reliable, efficient
tokenizing and parsing software.  Furthermore, the theoretical
underpinnings of tokenizing and parsing serve to identify ambiguity
that must be removed from a language description.

\subsection{Some Implications of Using Computer-Processable SGML}

Two points must be emphasized as a result of our call for
machine-processable DTD's.  The first of these is that rendering
DTD's, and instances of DTD's, processable by computers may render
them less readable or less understandable by humans. This should not
be viewed as an obstacle to development by humans, however, since
human concerns can be taken into account by applying various {\em
filters} to the machine-processable form.  For example, a
user-interface for creating instances of DTD's may provide a
human-friendly view to the human user, using short or omitted tags,
say, but a machine-processable, rigorously marked-up view for
application software.  We highly recommend separating considerations
regarding readability or usability by machine and by humans in order
to control the complexity of the definition and analysis of DTD's and
their instances.

The second point to be raised is that sufficient computer-science
expertise must be present in order to meet and maintain the goals of
machine-processability of DTD's.  The TEI must undertake to put such
expertise in place before commencement of the official definition of
TEI DTD's.

\section{Guidelines for Writing DTD's}

We have considerable previous experience with parsing \cite{kn:KAE88}
and translating \cite{kn:MAM87c} SGML DTD's.  In this section we
describe the guidelines that must be followed in the creation of SGML
DTD's if they are to be processable by existing software tools.  We
justify the guidelines only with minimal discussion and motivation.
For a detailed justification the reader should consult the sources
cited above or their authors.

\begin{itemize}

\item Use only the markup prescribed for {\em minimal} SGML documents.
In particular, short references and the omit tag feature of SGML are
not used in these documents. The use of short references and omit tags
call for special-purpose processing in both the token-recognition and
parsing stages of evaluating an SGML DTD and make them unsuitable for
straight-forward processing by tokenizer and parser generators.
Filters can be used to generate and (re)specify omittable tags, once
the grammar is in place.  Similarly, a second, manual pass can be
executed to indicate acceptable tag abbreviation conventions.

\item Use attributes only when information must be specified outside of the
hierarchical structure of the DTD.  The tokenizing and parsing of
attributes is different from that of tags and a separate processing
mechanism must be employed when attributes are used in the DTD
specification.  Thus, they should be used only when absolutely
required.  Tags already provide a mechanism for specifying all
hierarchical structure in a document, and they should be used for this
purpose.  If information must be specified that is not within the
hierarchical structure  (e.g., the relationship between two parts of
the verb in some German sentences), then attributes may be used.

\item Do not use inclusion and exclusion exceptions in the
DTD grammar.  For parser generators, such exceptions must be tediously
specified on a production-by-production basis, although a tool might
be implemented to do this automatically.  More importantly, however,
our experience indicates that (1) when the inclusion exception
mechanism is used, elements are included where they are really not
appropriate, and (2) when the exclusion exception mechanism is used
not only do grammars grow inordinately, and perhaps unnecessarily,
large, but unintended constructs are likely to occur.

\item Use principles of modular design to guide the specification of
the DTD. Human comprehension of these grammars is limited and as they
become complex unintended constructs may arise. In general, it is
better to employ many simple subgrammars that capture the unique
content of some content than to specify a few, large subgrammars that
become unwieldy and go beyond human comprehension.  Specific
guidelines in this case include:

\begin{itemize}

\item Use only 20-30 productions in a grammar.  If
there is a need for a grammar to become much larger than this,
subgrammars should likely be used.

\item  The tree represented by the grammar should be only 4-5 levels
deep.  Again, consider subgrammars if depth is much beyond this.

\item Define only simple content models (i.e., subgrammars) and use
them with care.

\end{itemize}

\item  If there is more than one way to encode a dataset, specify a
separate DTD for each application, using the {\em concur} feature of
SGML. For example, if one analysis package needs to recognize
formatting information in the data and another needs to recognize
structural information, then two separate DTD's should be written.
Trying to combine these separate encodings into one DTD will lead to
unnecessary complexity.

\end{itemize}

\section{Guidelines for Writing Instances of DTD's}

Given that DTD's are defined according to the above guidelines,
certain style conventions should be imposed when creating instances of
DTD's to further ensure ease of machine processing.

\begin{itemize}

\item When specifying attribute values in defining an instance of a
DTD, use the style {\em name=`value'}.  The {\em name} and {\em vi},
the value indicator or equal sign in our example, must be explicitly
written and the value of the attribute must be enclosed in single
quote marks.  If the value itself contains single quote marks, entity
references must be used to represent those characters in the value
string.  This guideline ensures an unambiguous interpretation of
attributes and their respective values.

\item When entering tags into the dataset, only the angle brackets and
the enclosed information, i.e., the tag names and any attributes,
should be specified.  No other extraneous {\em white space} characters
like tabs, carriage returns, line feeds and so on may be added outside
of the angle brackets.  This guideline is necessary to guarantee that
no new characters are introduced into the dataset itself during
encoding of the dataset.

\end{itemize}

\bibliography{/n/fruit/0/mamrak/publications/ieee-tse/cham}
\bibliographystyle{plain}

\end{document}
