Add a "Do's, don'ts & things to keep in mind" subsection to the
"Writing Tests" documentation. Much of this is based on Junio C
Hamano's "Test your stuff" section in
<7vhbkj2kcr.fsf@alter.siamese.dyndns.org>.
I turned it into a list of do's and don'ts to make it easier to skim
it, and integrated my note that a TAP harness will get confused if you
print "ok" or "not ok" at the beginning of a line.
t/README: Document the prereq functions, and 3-arg test_*
There was no documentation for the test_set_prereq and
test_have_prereq functions, or the three-arg form of
test_expect_success and test_expect_failure.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Brandon Casey [Mon, 28 Jun 2010 22:51:02 +0000 (17:51 -0500)]
t/t9700/test.pl: don't access private object members, use public access methods
This test is accessing private object members of the Test::More and
Test::Builder objects. Older versions of Test::More did not implement
these variables using a hash.
My system complains as follows:
Can't coerce array into hash at <snip>/t/t9700/test.pl line 13.
BEGIN failed--compilation aborted at <snip>/t/t9700/test.pl line 15.
There are public access methods available for retrieving and setting these
variables, so let's use them instead.
Signed-off-by: Brandon Casey <casey@nrlssc.navy.mil> Acked-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t9700: Use Test::More->builder, not $Test::Builder::Test
$Test::Builder::Test was only made into an `our' variable in 0.94
released in September 2009, older distros are more likely to have 0.92
or earlier. Use the singleton Test::More->builder constructor instead.
The exit() call was also unportable to <0.94. Just output a meaningful
exit code if the ->is_passing method exists. The t9700-perl-git.sh
test only cares about stderr output, so this doesn't affect test
results when using older Test::More modules.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: Say "pass" rather than "ok" on empty lines for TAP
Lines that begin with "ok" confuse the TAP harness because it can't
distinguish them from a test counter. Work around the issue by saying
"pass" instead, which isn't a reserved TAP word.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
tests: Skip tests in a way that makes sense under TAP
SKIP messages are now part of the TAP plan. A TAP harness now knows
why a particular test was skipped and can report that information. The
non-TAP harness built into Git's test-lib did nothing special with
these messages, and is unaffected by these changes.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
test-lib: output a newline before "ok" under a TAP harness
Some tests in the testsuite will emit a line that doesn't end with a
newline, right before we're about to output "ok" or "not ok". This
breaks the TAP output with "Tests out of sequence" errors since a TAP
harness can't understand this:
ok 1 - A test
[some output here]ok 2 - Another test
ok 3 - Yet another test
Work around it by emitting an empty line before we're about to say
"ok" or "not ok", but only if we're running under --verbose and
HARNESS_ACTIVE=1 is set, which'll only be the case when running under
a harnesses like prove(1).
I think it's better to do this than fix each tests by adding `&& echo'
everywhere. More tests might be added that break TAP in the future,
and a human isn't going to look at the extra whitespace, since
HARNESS_ACTIVE=1 always means a harness is reading it.
test-lib: Make the test_external_* functions TAP-aware
Before TAP we just ran the Perl test and assumed that it failed if
nothing was printed on STDERR. Continue doing that, but introduce a
`test_external_has_tap' variable which tests can set to indicate that
they're outputting TAP.
If it's set we won't output a test plan, but trust the external test
to do so. That way we can make external tests work with a TAP harness,
but still maintain compatibility with test-lib's own way of tracking
tests through the test-results directory.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
TAP, the Test Anything Protocol, is a simple text-based interface
between testing modules in a test harness. test-lib.sh's output was
already very close to being valid TAP. This change brings it all the
way there. Before:
$ ./t0005-signals.sh
* ok 1: sigchain works
* passed all 1 test(s)
And after:
$ ./t0005-signals.sh
ok 1 - sigchain works
# passed all 1 test(s)
1..1
The advantage of using TAP is that any program that reads the format
(a "test harness") can run the tests. The most popular of these is the
prove(1) utility that comes with Perl. It can run tests in parallel,
display colored output, format the output to console, file, HTML etc.,
and much more. An example:
prove(1) gives you human readable output without being too
verbose. Running the test suite in parallel with `make test -j15`
produces a flood of text. Running them with `prove -j 15 ./t[0-9]*.sh`
makes it easy to follow what's going on.
All this patch does is re-arrange the output a bit so that it conforms
with the TAP spec, everything that the test suite did before continues
to work. That includes aggregating results in t/test-results/, the
--verbose, --debug and other options for tests, and the test color
output.
TAP harnesses ignore everything that they don't know about, so running
the tests with --verbose works:
test_expect_failure 'true test (todo)' 'true'
test_expect_failure 'false test (todo)' 'false'
test_debug 'echo "debug message"'
test_done
The output of that was previously:
* diagnostic message # yellow
* ok 1: true test
* FAIL 2: false test # bold red
false
* FIXED 3: true test (todo)
* still broken 4: false test (todo) # bold green
* fixed 1 known breakage(s) # green
* still have 1 known breakage(s) # bold red
* failed 1 among remaining 3 test(s) # bold red
But is now:
diagnostic message # yellow
ok 1 - true test
not ok - 2 false test # bold red
# false
ok 3 - true test (todo) # TODO known breakage
not ok 4 - false test (todo) # TODO known breakage # bold green
# fixed 1 known breakage(s) # green
# still have 1 known breakage(s) # bold red
# failed 1 among remaining 3 test(s) # bold red
1..4
All the coloring is preserved when the test is run manually. Under
prove(1) the test performs as expected, even with --debug and
--verbose options:
$ prove ./example.sh :: --debug --verbose
./example.sh .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/4 subtests
(1 TODO test unexpectedly succeeded)
Junio C Hamano [Wed, 23 Jun 2010 17:27:39 +0000 (10:27 -0700)]
url.c: "<scheme>://" part at the beginning should not be URL decoded
When using the protocol git+ssh:// for example we do not want to
decode the '+' as a space. The url decoding must take place only
for the server name and parameters.
Junio C Hamano [Tue, 22 Jun 2010 16:45:21 +0000 (09:45 -0700)]
Merge branch 'cc/cherry-pick-series'
* cc/cherry-pick-series:
Documentation/revert: describe passing more than one commit
Documentation/cherry-pick: describe passing more than one commit
revert: add tests to check cherry-picking many commits
revert: allow cherry-picking more than one commit
revert: change help_msg() to take no argument
revert: refactor code into a do_pick_commit() function
revert: use run_command_v_opt() instead of execv_git_cmd()
revert: cleanup code for -x option
Junio C Hamano [Tue, 22 Jun 2010 16:45:21 +0000 (09:45 -0700)]
Merge branch 'jc/rev-list-ancestry-path'
* jc/rev-list-ancestry-path:
revision: Turn off history simplification in --ancestry-path mode
revision: Fix typo in --ancestry-path error message
Documentation/rev-list-options.txt: Explain --ancestry-path
Documentation/rev-list-options.txt: Fix missing line in example history graph
revision: --ancestry-path
Junio C Hamano [Tue, 22 Jun 2010 15:31:25 +0000 (08:31 -0700)]
Merge branch 'tc/merge-m-log' into maint
* tc/merge-m-log:
merge: --log appends shortlog to message if specified
fmt-merge-msg: add function to append shortlog only
fmt-merge-msg: refactor merge title formatting
fmt-merge-msg: minor refactor of fmt_merge_msg()
merge: rename variable
merge: update comment
t7604-merge-custom-message: show that --log doesn't append to -m
t7604-merge-custom-message: shift expected output creation
Junio C Hamano [Tue, 22 Jun 2010 15:30:38 +0000 (08:30 -0700)]
Merge branch 'pc/remove-warn' into maint
* pc/remove-warn:
Remove a redundant errno test in a usage of remove_path
Introduce remove_or_warn function
Implement the rmdir_or_warn function
Generalise the unlink_or_warn function
Junio C Hamano [Mon, 21 Jun 2010 13:02:47 +0000 (06:02 -0700)]
Merge branch 'sm/branch-broken-ref'
* sm/branch-broken-ref:
branch: don't fail listing branches if one of the commits wasn't found
branch: exit status now reflects if branch listing finds an error
Junio C Hamano [Mon, 21 Jun 2010 13:02:45 +0000 (06:02 -0700)]
Merge branch 'js/async-thread'
* js/async-thread:
fast-import: die_nicely() back to vsnprintf (reverts part of ebaa79f)
Enable threaded async procedures whenever pthreads is available
Dying in an async procedure should only exit the thread, not the process.
Reimplement async procedures using pthreads
Windows: more pthreads functions
Fix signature of fcntl() compatibility dummy
Make report() from usage.c public as vreportf() and use it.
Modernize t5530-upload-pack-error.
Junio C Hamano [Mon, 21 Jun 2010 13:02:44 +0000 (06:02 -0700)]
Merge branch 'gv/portable'
* gv/portable:
test-lib: use DIFF definition from GIT-BUILD-OPTIONS
build: propagate $DIFF to scripts
Makefile: Tru64 portability fix
Makefile: HP-UX 10.20 portability fixes
Makefile: HPUX11 portability fixes
Makefile: SunOS 5.6 portability fix
inline declaration does not work on AIX
Allow disabling "inline"
Some platforms lack socklen_t type
Make NO_{INET_NTOP,INET_PTON} configured independently
Makefile: some platforms do not have hstrerror anywhere
git-compat-util.h: some platforms with mmap() lack MAP_FAILED definition
test_cmp: do not use "diff -u" on platforms that lack one
fixup: do not unconditionally disable "diff -u"
tests: use "test_cmp", not "diff", when verifying the result
Do not use "diff" found on PATH while building and installing
enums: omit trailing comma for portability
Makefile: -lpthread may still be necessary when libc has only pthread stubs
Rewrite dynamic structure initializations to runtime assignment
Makefile: pass CPPFLAGS through to fllow customization
Junio C Hamano [Mon, 21 Jun 2010 13:02:42 +0000 (06:02 -0700)]
Merge branch 'bc/portable'
* bc/portable:
Remove python 2.5'isms
Makefile: add PYTHON_PATH to GIT-BUILD-OPTIONS
t/aggregate-results: accomodate systems with small max argument list length
t/t7006: ignore return status of shell's unset builtin
t/t5150: remove space from sed script
git-request-pull.sh: remove -e switch to shell interpreter which breaks ksh
t/t5800: skip if python version is older than 2.5
Junio C Hamano [Mon, 21 Jun 2010 13:02:42 +0000 (06:02 -0700)]
Merge branch 'jn/gitweb-fastcgi'
* jn/gitweb-fastcgi:
gitweb: Run in FastCGI mode if gitweb script has .fcgi extension
gitweb: Add support for FastCGI, using CGI::Fast
gitweb: Put all per-connection code in run() subroutine
git-cvsserver: fix error for invalid password formats
Change the error message to report the erroneous password
character. $1 was never set in the previos version, it was a leftover
from older code that used a regex for the test.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Fri, 18 Jun 2010 18:16:57 +0000 (11:16 -0700)]
Merge branch 'by/diff-graph'
* by/diff-graph:
Make --color-words work well with --graph
graph.c: register a callback for graph output
Emit a whole line in one go
diff.c: Output the text graph padding before each diff line
Output the graph columns at the end of the commit message
Add a prefix output callback to diff output
Junio C Hamano [Fri, 18 Jun 2010 18:16:55 +0000 (11:16 -0700)]
Merge branch 'jn/gitweb-plackup'
* jn/gitweb-plackup:
git-instaweb: Add support for running gitweb via 'plackup'
git-instaweb: Wait for server to start before running web browser
git-instaweb: Remove pidfile after stopping web server
git-instaweb: Configure it to work with new gitweb structure
git-instaweb: Put httpd logs in a "$httpd_only" subdirectory
gitweb: Set default destination directory for installing gitweb in Makefile
gitweb: Move static files into seperate subdirectory
Junio C Hamano [Fri, 18 Jun 2010 18:16:54 +0000 (11:16 -0700)]
Merge branch 'tc/merge-m-log'
* tc/merge-m-log:
merge: --log appends shortlog to message if specified
fmt-merge-msg: add function to append shortlog only
fmt-merge-msg: refactor merge title formatting
fmt-merge-msg: minor refactor of fmt_merge_msg()
merge: rename variable
merge: update comment
t7604-merge-custom-message: show that --log doesn't append to -m
t7604-merge-custom-message: shift expected output creation