Encourage having external tests in multiple test recipes

This will make the individual external tests more easily selectable /
deselectable through the usual test selection mechanism.

This also moves external tests to group 95.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2902)

diff --git a/test/README.external b/test/README.external
index 3cd0541408af70a48273755877ad1b158945d53e..f411e731ec4ea81c90a033f315c76492129fe4cd 100644 (file)
--- a/test/README.external
+++ b/test/README.external
@@ -1,13 +1,17 @@
-Running the BoringSSL test suite with OpenSSL
-=============================================
-
-It is possible to integrate external test suites into OpenSSL's "make test". At
-the current time the only supported external suite is the one used by
-BoringSSL.
+Running external test suites with OpenSSL
+=========================================
 
+It is possible to integrate external test suites into OpenSSL's "make test".
 This capability is considered a developer option and may not work on all
 platforms.
 
+At the current time the only supported external suite is the one used by
+BoringSSL.
+
+
+The BoringSSL test suite
+========================
+
 In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL
 source code into an appropriate directory:
 
@@ -40,11 +44,11 @@ To see more detailed output you can run just the BoringSSL tests with the
 verbose option:
 
 $ VERBOSE=1 BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make \
-  TESTS="test_external" test
+  TESTS="test_external_boringssl" test
 
 
 Test failures and suppressions
-==============================
+------------------------------
 
 A large number of the BoringSSL tests are known to fail. A test could fail
 because of many possible reasons. For example:

diff --git a/test/recipes/90-test_external.t b/test/recipes/95-test_external_boringssl.t
similarity index 71%
rename from test/recipes/90-test_external.t
rename to test/recipes/95-test_external_boringssl.t
index 3bdc4afe67d14793c190f29dd74c68087dcb092d..a49e6ea466eb07e2c190d9ce2e01bcb259ee0d0f 100644 (file)
--- a/test/recipes/90-test_external.t
+++ b/test/recipes/95-test_external_boringssl.t
@@ -11,24 +11,23 @@ use OpenSSL::Test;
 use OpenSSL::Test::Utils;
 use OpenSSL::Test qw/:DEFAULT bldtop_file srctop_file cmdstr/;
 
-setup("test_external");
+setup("test_external_boringssl");
 
 plan skip_all => "No external tests in this configuration"
     if disabled("external-tests");
 
-if (!$ENV{BORING_RUNNER_DIR}) {
-    plan skip_all => "No external tests have been detected";
-}
+plan skip_all => "BoringSSL runner not detected"
+    if !$ENV{BORING_RUNNER_DIR};
 
 plan tests => 1;
 
 indir $ENV{BORING_RUNNER_DIR} => sub {
     ok(filter_run(cmd(["go", "test", "-shim-path",
-                      bldtop_file("test", "ossl_shim", "ossl_shim"),
-                      "-shim-config",
-                      srctop_file("test", "ossl_shim", "ossl_config.json"),
-                      "-pipe", "-allow-unimplemented"])),
-        "running external tests");
+                       bldtop_file("test", "ossl_shim", "ossl_shim"),
+                       "-shim-config",
+                       srctop_file("test", "ossl_shim", "ossl_config.json"),
+                       "-pipe", "-allow-unimplemented"])),
+       "running BoringSSL tests");
 }, create => 0, cleanup => 0;
 
 # Filter the output so that the "ok" printed by go test doesn't confuse