TCW09: Backward Compatibility and the Maintenance of the Text Encoding Initiative Guidelines



At the May 2006 TEI Council Meeting in Kyoto, Council expressed an interest in elaborating principles for further development of the TEI guidelines. In particular, Council expressed an interest in reconciling two potentially conflicting principles:
  1. As new technologies permit us to improve the ability of the TEI guidelines to serve the research needs of users, those new capabilities should be made available to users. The TEI should not eschew better ways of doing things for fear of breaking backward compatibility.
  2. Because breaking backward compatibility is unpopular with users, and for good reason, the TEI should avoid doing so unless the benefits are compelling.

Some Council members argued in Kyoto that our license to break backward compatibility would be revoked once we signed off on P5; others felt that Council (especially new Council members) would appreciate the availability of guiding and stable principles that did not simply prohibit breaking backward compatibility, but that indicated how and when it should be permitted.

It should be noted that documents developed with a particular schema remain valid against that schema in perpetuity, and in this sense no change to the TEI Guidelines breaks backward compatibility. On the other hand, a user may wish to integrate new, emerging features into an existing project without rendering existing encoded information invalid, and may need to produce a new schema for that purpose. It is in this context that backward compatibility with existing features of the user's original schema becomes particularly important.

Finally, the TEI currently seems to make no use of the concept of deprecated features, although such a concept may provide a compromise that allows the TEI to adopt new structures without invalidating old ones.


Relevant sources for determining a suitable policy include:

None of these documents explicitly prohibits the breaking of backward compatibility, although TEI EDW57 distinguishes corrigible errors from those that are not corrigible according to (among other things) whether correction would break backward compatibility. The editors are empowered to implement changes needed to remove corrigible errors without further ado; if not, the proposed correction must be "referred to TC" (i.e. to the "Technical [Review] Committee," or presumably now the TEI Technical Council) for action. The historical record thus seems to suggest an awareness that breaking backward compatibility might confer sufficient benefit that it should not be prohibited, but also that it should not be undertaken without careful review.

Proposed Guidelines

A. Council should not be constrained by a policy that bluntly prohibits breaking backward compatibility after the publication of P5. Council should instead feel authorized to break backward compatibility when Council concludes that the advantages of doing so outweigh the disadvantages.

B. Wherever possible, Council should implement changes so that new schemas are supersets of old ones, and do not render invalid any documents that would have been valid before the implementation of the changes. The addition of an element to a class creates such a superset; removing an element from a class has the opposite effect.

C. In extreme situations, where Council concludes that the utility of a component would be improved substantially by a revision that could break backward compatibility, Council should:
  1. Create the necessary new structure (class, content model, etc.) under a new name.
  2. Deprecate but retain the structure it is intended to replace.
    • Deprecated structures determined to be in wide use might be maintained in perpetuity. Such a decision would reflect Council's expert opinion that eliminating them would be too disruptive to existing projects to be entertained, while making clear that their employment in new projects does not conform to best practice.
    • Deprecated structures determined not to be in wide use might eventually (as discussed immediately below) be removed from the Guidelines.
  3. Council may break backward compatibility in a more substantial and comprehensive way (as happened with the transition to P4 to P5) only with the transition to a new major version (e.g., P6, but not P5.99999). Such transitions should happen only in response to the emergence of new technologies (e.g., the transition from SGML to XML) or the development of a new architecture that conveys substantial benefit (e.g., the development of the new class system). Historically, new major versions have emerged approximately every five years.