TCW31: Update P5 subset

TEI Council: Update P5 subset

Rationale: the Stylesheets are built against the Guidelines, so there needs to be a (more or less) up-to-date copy of the Guidelines in the Stylesheet repository. This copy of the Guidelines, the file “p5subset.xml”, is updated monthly by Council members.

Step-by-Step Instructions:

Get the p5subset.xml (two possibilities):

from a fresh build of the P5 dev branch (preferably on Jenkins, at a URL such as https://jenkins.tei-c.org/job/TEIP5-dev/lastSuccessfulBuild/artifact/P5/release/xml/tei/odd/p5subset.xml)

build the p5subset locally

use your local copy of the P5 dev branch or install via docker

start the TEI docker container

change to the [relative path to TEIC/TEI/ directory]/P5 directory

run make test

after you finish the steps below, don’t forget to clean your directory with make clean to make sure that files generated while running the test are removed again

Change to the Stylesheets dev branch

Update the version of the p5subset.xml in the Stylesheets/source/ directory in the Stylesheets dev branch: cp -p [relative path to TEIC/TEI/ directory]/P5/p5subset.xml source/p5subset.xml. For example, if you have the TEIC/TEI/ repo in ~/TEICouncil/repos/TEI/ and the TEIC/Stylesheets repo in ~/TEICouncil/repos/Stylesheets/, you would issue:

$ cd ~/TEICouncil/Stylesheets/

$ cp -p ../P5/p5subset.xml source/p5subset.xml

Run Test2 to make sure that the results are as expected (run in the docker image if you are using the docker approach):

$ cd Test2

$ ant test

Error handling in Test2 (if there are no errors from Test2 process, go on to Test, below.):

The vast majority of all errors from Test2 will be “diff errors”, i.e. a difference between a file generated from processing with the new p5subset (in the Test2/outputFiles/ directory) and the corresponding file that had previously been generated from processing with the old p5subset (in the expected-results/ directory).

If the process stops with an error that is not a diff error, notice whether it's due to something failing to load that the testing process requires. If that's the case, read the error message carefully and see if you can figure out what's failing (and reach out to Council members for help). See Troubleshooting, below for an example failure of the ant process and a simple fix.

In the case of a diff error, examine the differences generated. If the differences are what you would expect given the change in P5 (which is by far the most common case), just copy the output file to be the new expected results file. For example, if one of the changes made to P5 was to add an English gloss for mentioned, a diff error would be entirely expected. It would look like

 [echo] about to compare files: [echo] inFile otherFile =
                      [path]/Test2/outputFiles/testPure1.rng
                      [path]/Test2/expected-results/testPure1.rng [echo] ERROR: DIFF
                      FAILURE… [exec] output: <a:documentation
                      xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0">(mentioned)
                      contains a specialized form of heading or label, giving the name of one or
                      more speakers in a dramatic text or fragment. [3.13.2. Core Tags for
                      Drama]</a:documentation> [exec] expect: <a:documentation
                      xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0">contains a
                      specialized form of heading or label, giving the name of one or more speakers
                      in a dramatic text or fragment. [3.13.2. Core Tags for
                      Drama]</a:documentation> [exec] Result: 1 BUILD FAILED
                      [path]/Test2/build.xml:541: The following error occurred while executing this
                      line: [path]/Test2/build_odd.xml:44: The following error occurred while
                      executing this line: [path]/Test2/build_odd.xml:103: The following error
                      occurred while executing this line: [path]/Test2/build_utilities.xml:148:
                      The following error occurred while executing this line:
                      [path]/Test2/build_utilities.xml:210: Build failed because of differences
                      between [path]/Test2/outputFiles/testPure1.rng and
                      [path]/Test2/expected-results/testPure1.rng. See diff output above.

As a human you can quickly look at the differences (“(mentioned)” was inserted), and realize that is an appropriate change. So to fix this error you just copy the actual output file to be the new expected file. Note that the 2nd line of the output is specifically designed to make executing the desired copy command easy. (You can copy everything after the “=”, type “cp -p” on the command line and then paste in the paths: $ cp -p [path]/Test2/outputFiles/testPure1.rng [path]/Test2/expected-results/testPure1.rng Note that the “-p” switch is optional — it just gives the copy the same timestamp and permissions as the original.) If the error is either a diff error you would not have expected, or worse a different kind of error completely, fix it. Note that fixing it might be trivially easy or might take weeks of work from half a dozen different people.

If the error is either a diff error you would not have expected, or worse a different kind of error completely, fix it. Note that fixing it might be trivially easy or might take weeks of work from half a dozen different people.

After all errors have been fixed in Test2/, move on to Test/.

switch to the main directory (cd .. will do, if you are still in Test2/ from previous step)

run either make or time make [See Addendum, below, for using the –jobs switch to make make run faster.]

check the errors (the make file stops after each error)

When there is an error, you will find a diff of the relevant file from the actual-results/ folder and the expected-results/ folder.

In case the output is not as expected (i.e., the difference is a real problem, rather than just an expected difference from changes made to p5subset), fix the error.

In case the output is as expected, copy the file from actual-results/ to expected-results/. As with the Test2/ case, you can copy and paste the correct paths from the error message. It looks like “then diff actual-results/test.rng expected-results/test.rng;”. You just need to replace the initial “then diff” with “cp -p” and strip off the ending semicolon. Thus:

$ cp -p Test/actual-results/test.rng

Test/expected-results/test.rng

If you are quite comfortable on the commandline and facile with a text editor, you might prefer to run all the tests in Test/ at once and test the outputs yourself, rather than have the make command test the outputs, because it stops after the first error. (Remember that a lot of what the Makefile does is transform a test file using the Stylesheets and then compare the actual output of that command to a file which contains the expected output of that command. These comparisons are done using the diff command.) If you ask it nicely, the make command will just generate the outputs, and defer the actual testing of them (by diffing them with the corresponding expected output). This means that running make is dramatically faster, but it does not do all the work, you have to do some of it. To do this:

Make sure you are in the Test/ directory.

$ make DIFFNOW=0

or, if you want to try to use multiple threads:

$ make DIFFNOW=0 --jobs=`nproc 2>/dev/null || echo 1` -Oline

To actually see the filenames being diffed, add the “-C0” switch:

make DIFFNOW=0 -C0

When the Makefile has run a transformation, instead of comparing the actual output of that transformation to the expected output, it will say something like “==deferring: ` diff actual-results/test27.html expected-results/test27.html `”.

Once the make command is complete, you now need to perform all those comparisons yourself. Luckily, this is designed to be relatively easy. E.g., each message that reports a diff command that has not been executed is preceded with “==”, and no other output from the Makefile is.

Copy the output from the make DIFFNOW=0 -C0 command into your favorite text editor.

Delete all lines that do not start with “==”.

Remove the “==deferring: `” from the beginning of each line.

Remove the “`” from the end of each line.

Insert “#! /bin/bash” as a 1st line.

Save this file as diffnow_erase_me_soon.bash, or whatever.

Change the mode of this new file to be executable (i.e., chmod a+x diffnow_erase_me_soon.bash).

Run it (i.e. ./diffnow_erase_me_soon.bash).

Addendum: faster testing (recommendations by Syd Bauman)

One of the reasons the test procedure in Stylesheets/Test2/ is dramatically faster than the one in Stylesheets/Test/ is that it is, by default, run in parallel.

(ant test runs them in parallel; if you want them in series (likely because the order of messages was confusing when run in parallel) use ant testSeries.)

(There are other reasons, like it is written to be less redundant, and the JVM is only spun up once, rather than once for every test.)

You can also ask the make command to run multiple jobs at once. The switches that control this are --jobs= and --output-sync=. I just tried an experiment, comparing how long it took to run make vs make --jobs=7 --output-sync=lines. (I chose 7 because my system has 8 threads, and I wanted to have some CPU available. What little I have found on the web seems to suggest I may as well go ahead and use 8.)

The result was faster, although not even close to 7 times faster: down to 03:32 from 04:36. I compared the output of the 2 commands, and they were identical.

On GNU/Linux, at least, the nproc command will tell you how many threads are available. Thus using the command

$ make --jobs=`nproc 2>/dev/null || echo 1` -Oline

seems to make sense to me. (-O is shorthand for --output-sync=.)

It is also possible to get the Makefile to do that on its own. My first thought is that might not be such a good idea, because you may want to run with --jobs=1 in order to force error messages into the right order. (I.e., in case -Oline wasn’t good enough.)

I ran the experiment again, this time using --jobs=8 and getting screen captures of the process monitor roughly 40 s after the make command started. The timing results were very similar (down to 03:36 from 04:36), but the order of output lines was different. (Same output; i.e., they were identical after sorting and removing timestamps.)

So I think anyone running the Stylesheets test process would do well to use the --jobs switch. You could use any of

 -j 8 # if you know you have 8
              threads, e.g. --jobs=`nproc` # if you know the nproc command works on your
              system -j `nproc 2> /dev/null || echo 1` # if there is a chance nproc
              fails, # so it defaults to '1'

Troubleshooting

Test 2: What if the ant process fails because some necessary dependency is missing? For example, you may see an error message like this:

A class needed by class org.apache.fop.tools.anttasks.Fop cannot be found:
              org/apache/commons/logging/Log using the classloader
              AntClassLoader[/YOUR/FILEPATH/TO/TEIC/Stylesheets/lib/fop-2.6/fop/lib/jeuclid-core-3.1.9.jar:/YOUR/FILEPATH/TO/TEIC/Stylesheets/lib/fop-2.6/fop/lib/jeuclid-fop-3.1.9.jar:/Users/eeb4/Documents/GitHub/TEIC/Stylesheets/lib/fop-2.6/fop/build/fop-hyph.jar:/Users/eeb4/Documents/GitHub/TEIC/Stylesheets/lib/fop-2.6/fop/build/fop.jar

This signals that a dependency is missing or corrupted. In our example, fop-2.6 is a directory generated by ant while it is running a jar dependency. Perhaps a network connection was interrupted somehow or the process didn't complete as it was supposed to do. The simplest way to correct this is to delete the fop-2.6 directory, and return to Test2 and run “ant test” again. This lets ant pull in a clean copy of the missing dependency.