document new suite layout

diff --git a/tests/README b/tests/README
index f948eaf3d29b1e49c4c030a0dc1a9255c045f075..43387f46758804d919781745f754c3276f10219c 100644 (file)
--- a/tests/README
+++ b/tests/README
@@ -8,14 +8,14 @@ this directory and its contents.
 
 * STRUCTURE OF THE TEST SUITE
 
-The test suite consists of several directories, each containing a
-scanner known to work with the most recent version of flex. In
+The testsuite consists of several tests. Each test is centered around
+a scanner known to work with the most recent version of flex. In
 general, after you modify your copy of the flex distribution, you
-should re-run the test suite. Some of the tests may require certain tools
-to be available (e.g., bison, diff). If any test returns an error or
-generates an error message, then your modifications *may* have broken
-a feature of flex. At a minimum, you'll want to investigate the
-failure and determine if it's truly significant.
+should re-run the test suite. Some of the tests may require certain
+tools to be available (e.g., bison, diff). If any test returns an
+error or generates an error message, then your modifications *may*
+have broken a feature of flex. At a minimum, you'll want to
+investigate the failure and determine if it's truly significant.
 
 * HOW TO RUN THE TEST SUITE
 
@@ -26,52 +26,32 @@ To build and execute all tests from the top level of the flex source tree:
 To build and execute a single test:
 
   $ cd tests/ # from the top level of the flex tree.
-  $ cd test-pthread-nr # for example
-  $ make test
+  $ make testname.log
 
-* HOW TO ADD A NEW TEST TO THE TEST SUITE
+  where "testname" is the name of the test. This is an automake-ism
+  that will create (or re-create, if need be), a log of the particular
+  test run that you're working on.
 
-**- RUN the script `create-test` found in this directory with a single
-argument of the name of the test you want to create.  If it fails with
-a message about a non-existent file `config.status', then run the
-configure script in the top-level directory and everything will be
-fine.  If it fails to work for you other than this, report it as a
-bug.
+* HOW TO ADD A NEW TEST TO THE TEST SUITE
 
-** Modify the files in the newly created directory so that they test
-whatever feature of flex you are interested in.
+** List your test in the TESTS variable in Makefile.am in this
+   directory. Note that due to the large number of tests, we use
+   variables to group similar tests together. This also helps with
+   handling the automake test suite requirements. Hopefully your test
+   can be listed in SIMPLE_TESTS. You'll need to add the appropriate
+   automake _SOURCES variable as well. If you're unsure, then consult
+   the automake manual, paying attention to the parallel test harness
+   section.
 
 ** On success, your test should return zero.
 
 ** On error, your test should return 1 (one) and print a message to
-stderr, which will have been redirected to the file named, "OUTPUT",
-in your test's directory.
+stderr, which will have been redirected to the log file created by the
+automake test suite harness.
 
 ** If your test is skipped (e.g., because bison was not found), then
-   the test should return 2 (two). See "test-bison-nr/Makefile.am" for
-   an example.
-
-** You must modify the last few lines of the top-level configure.in by
-   adding the Makefile for your test directory.  (This step is
-   done automatically by `create-test`.)
-
-** You must add the name of your test to the SUBDIRS variable in
-   tests/Makefile.am. (This is also done automatically for you by `create-test'.)
-
-** Add a description of your new test to the end of the file
-    `descriptions'. Remember to keep the description as brief as
-    possible, preferably to one line.
-
-** You will have to run the autogen.sh script in the top-level
-   directory as well as run the configure script in that
-   directory. (Note that running config.status may prove easier or
-   quicker.)
-
-** The easiest way for you to submit your new test to the flex
-   maintainers is by generating a patch. The flex maintainers only
-   need to have the Makefile.am, the flex input file, the test input
-   file (if there is one) and any other files necessary to compile the
-   test. You do not need to submit files generated by autoconf,
-   automake, configure etc. It would be helpful to include the file
-   .cvsignore which you will find in your test directory if you used
-   the `create-test' script.
+   the test should return 2 (two).
+
+** Once your work is done, submit a patch via the flex development
+   mailing list, the github pull request mechanism or some other
+   suitable means.