Junio C Hamano [Mon, 30 Sep 2019 04:42:08 +0000 (13:42 +0900)]
Merge branch 'en/clean-nested-with-ignored' into next
"git clean" fixes.
* en/clean-nested-with-ignored:
clean: fix theoretical path corruption
clean: rewrap overly long line
clean: avoid removing untracked files in a nested git repository
clean: disambiguate the definition of -d
git-clean.txt: do not claim we will delete files with -n/--dry-run
dir: add commentary explaining match_pathspec_item's return value
dir: if our pathspec might match files under a dir, recurse into it
dir: make the DO_MATCH_SUBMODULE code reusable for a non-submodule case
dir: also check directories for matching pathspecs
dir: fix off-by-one error in match_pathspec_item
dir: fix typo in comment
t7300: add testcases showing failure to clean specified pathspecs
Junio C Hamano [Mon, 30 Sep 2019 04:42:06 +0000 (13:42 +0900)]
Merge branch 'dl/cocci-everywhere' into next
Coccinelle checks are done on more source files than before now.
* dl/cocci-everywhere:
Makefile: run coccicheck on more source files
Makefile: strip leading ./ in $(FIND_SOURCE_FILES)
Makefile: define THIRD_PARTY_SOURCES
Makefile: strip leading ./ in $(LIB_H)
Junio C Hamano [Mon, 30 Sep 2019 04:42:02 +0000 (13:42 +0900)]
Merge branch 'jk/disable-commit-graph-during-upload-pack' into next
The "upload-pack" (the counterpart of "git fetch") needs to disable
commit-graph when responding to a shallow clone/fetch request, but
the way this was done made Git panic, which has been corrected.
* jk/disable-commit-graph-during-upload-pack:
upload-pack: disable commit graph more gently for shallow traversal
commit-graph: bump DIE_ON_LOAD check to actual load-time
Junio C Hamano [Mon, 30 Sep 2019 04:42:00 +0000 (13:42 +0900)]
Merge branch 'rs/simplify-by-deco-with-deco-refs-exclude' into next
"git log --decorate-refs-exclude=<pattern>" was incorrectly
overruled when the "--simplify-by-decoration" option is used, which
has been corrected.
* rs/simplify-by-deco-with-deco-refs-exclude:
log-tree: call load_ref_decorations() in get_name_decoration()
log: test --decorate-refs-exclude with --simplify-by-decoration
Junio C Hamano [Mon, 30 Sep 2019 04:41:59 +0000 (13:41 +0900)]
Merge branch 'jk/partial-clone-sparse-blob' into next
The name of the blob object that stores the filter specification
for sparse cloning/fetching was interpreted in a wrong place in the
code, causing Git to abort.
* jk/partial-clone-sparse-blob:
list-objects-filter: use empty string instead of NULL for sparse "base"
list-objects-filter: give a more specific error sparse parsing error
list-objects-filter: delay parsing of sparse oid
t5616: test cloning/fetching with sparse:oid=<oid> filter
Junio C Hamano [Mon, 30 Sep 2019 04:41:58 +0000 (13:41 +0900)]
Merge branch 'ma/asciidoctor-refmiscinfo' into next
Update support for Asciidoctor documentation toolchain.
* ma/asciidoctor-refmiscinfo:
doc-diff: replace --cut-header-footer with --cut-footer
asciidoctor-extensions: provide `<refmiscinfo/>`
Doc/Makefile: give mansource/-version/-manual attributes
Using the pull command instead of push is more accurate when giving
instructions on placing the psuh command in alphabetical order.
Signed-off-by: Pedro Sousa <pedroteosousa@gmail.com> Acked-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor [Sat, 21 Sep 2019 07:40:54 +0000 (09:40 +0200)]
travis-ci: do not skip successfully tested trees in debug mode
Travis CI offers shell access to its virtual machine environment
running the build jobs, called "debug mode" [1]. After restarting a
build job in debug mode and logging in, the first thing I usually do
is to install dependencies, i.e. run './ci/install-dependencies.sh'.
This works just fine when I restarted a failed build job in debug
mode. However, after restarting a successful build job in debug mode
our CI scripts get all clever, and exit without doing anything useful,
claiming that "This commit's tree has already been built and tested
successfully" [2]. Our CI scripts are right, and we do want to skip
building and testing already known good trees in "regular" CI builds.
In debug mode, however, this is a nuisiance, because one has to delete
the cache (or at least the 'good-trees' file in the cache) to proceed.
Let's update our CI scripts, in particular the common 'ci/lib.sh', to
not skip previously successfully built and tested trees in debug mode,
so all those scripts will do what there were supposed to do even when
a successful build job was restarted in debug mode.
[1] https://docs.travis-ci.com/user/running-build-in-debug-mode/
[2] 9cc2c76f5e (travis-ci: record and skip successfully built trees,
2017-12-31)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor [Fri, 20 Sep 2019 09:36:09 +0000 (11:36 +0200)]
t/helper: ignore only executable files
This patch conceptually reverts 44103f4197 (t/helper: ignore
everything but sources, 2017-12-12). Back in those days we did have a
lot of separate test helper executables under 't/helper', and its
'.gitignore' did get out of sync every once in a while.
Since then, however, most of those separate executables were
integrated into a single 'test-tool' command [1], and new test helpers
are added as new subcommands, so the chances of that '.gitignore'
getting out of sync again are much lower. And even if a contributor
were not careful enough and submits a patch that adds a new executable
under 't/helper' but forgets to update '.gitignore' accordingly, our
CI builds would catch it in a timely manner [2].
Ignoring everything but sources has the drawback that building an
older version of Git (e.g. during bisecting) creates all those
executables, and after going back to e.g. current 'master' the usual
cleanup commands like 'make clean' or 'git clean -fd' don't remove
them (the former doesn't know about them, and the latter doesn't
remove ignored files).
So let's ignore only the executable files under 't/helper/, i.e.
'test-tool' and the three other remaining executables that could not
be integrated into 'test-tool' (no need to ignore object files, as
they are already ignored by our toplevel '.gitignore').
[1] The topic starting with efd71f8913 (t/helper: add an empty
test-tool program, 2018-03-24), and leading up to the merge commit 27f25845cf (Merge branch 'nd/combined-test-helper', 2018-04-11).
[2] b92cb86ea1 (travis-ci: check that all build artifacts are
.gitignore-d, 2017-12-31)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Thu, 19 Sep 2019 20:48:30 +0000 (22:48 +0200)]
git: use COPY_ARRAY and MOVE_ARRAY in handle_alias()
Use the macro COPY_ARRAY to copy array elements and MOVE_ARRAY to do the
same for moving them backwards in an array with potential overlap. The
result is shorter and safer, as it infers the element type automatically
and does a (very) basic type compatibility check for its first two
arguments.
These cases were missed by Coccinelle and contrib/coccinelle/array.cocci
because the type of the elements is "const char *", not "char *", and
the rules in the semantic patch cautiously insist on the sizeof operator
being used on exactly the same type to avoid generating transformations
that introduce subtle bugs into tricky code.
Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Gummerer [Wed, 11 Sep 2019 18:20:27 +0000 (19:20 +0100)]
stash: make sure to write refreshed cache
When converting stash into C, calls to 'git update-index --refresh'
were replaced with the 'refresh_cache()' function. That is fine as
long as the index is only needed in-core, and not re-read from disk.
However in many cases we do actually need the refreshed index to be
written to disk, for example 'merge_recursive_generic()' discards the
in-core index before re-reading it from disk, and in the case of 'apply
--quiet', the 'refresh_cache()' we currently have is pointless without
writing the index to disk.
Always write the index after refreshing it to ensure there are no
regressions in this compared to the scripted stash. In the future we
can consider avoiding the write where possible after making sure none
of the subsequent calls actually need the refreshed cache, and it is
not expected to be refreshed after stash exits or it is written
somewhere else already.
Reported-by: Jeff King <peff@peff.net> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Gummerer [Wed, 11 Sep 2019 18:20:26 +0000 (19:20 +0100)]
merge: use refresh_and_write_cache
Use the 'refresh_and_write_cache()' convenience function introduced in
the last commit, instead of refreshing and writing the index manually
in merge.c
Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Thomas Gummerer [Wed, 11 Sep 2019 18:20:25 +0000 (19:20 +0100)]
factor out refresh_and_write_cache function
Getting the lock for the index, refreshing it and then writing it is a
pattern that happens more than once throughout the codebase, and isn't
trivial to get right. Factor out the refresh_and_write_cache function
from builtin/am.c to read-cache.c, so it can be re-used in other
places in a subsequent commit.
Note that we return different error codes for failing to refresh the
cache, and failing to write the index. The current caller only cares
about failing to write the index. However for other callers we're
going to convert in subsequent patches we will need this distinction.
Helped-by: Martin Ågren <martin.agren@gmail.com> Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Thomas Gummerer <t.gummerer@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Wed, 18 Sep 2019 22:21:51 +0000 (15:21 -0700)]
Merge branch 'bw/rebase-autostash-keep-current-branch' into next
"git rebase --autostash <upstream> <branch>", when <branch> is
different from the current branch, incorrectly moved the tip of the
current branch, which has been corrected.
* bw/rebase-autostash-keep-current-branch:
builtin/rebase.c: Remove pointless message
builtin/rebase.c: make sure the active branch isn't moved when autostashing
Junio C Hamano [Wed, 18 Sep 2019 22:21:51 +0000 (15:21 -0700)]
Merge branch 'ds/include-exclude' into next
The internal code originally invented for ".gitignore" processing
got reshuffled and renamed to make it less tied to "excluding" and
stress more that it is about "matching", as it has been reused for
things like sparse checkout specification that want to check if a
path is "included".
* ds/include-exclude:
unpack-trees: rename 'is_excluded_from_list()'
treewide: rename 'exclude' methods to 'pattern'
treewide: rename 'EXCL_FLAG_' to 'PATTERN_FLAG_'
treewide: rename 'struct exclude_list' to 'struct pattern_list'
treewide: rename 'struct exclude' to 'struct path_pattern'
Junio C Hamano [Wed, 18 Sep 2019 22:21:50 +0000 (15:21 -0700)]
Merge branch 'en/merge-recursive-cleanup' into next
The merge-recursive machiery is one of the most complex parts of
the system that accumulated cruft over time. This large series
cleans up the implementation quite a bit.
* en/merge-recursive-cleanup: (24 commits)
merge-recursive: alphabetize include list
merge-recursive: add sanity checks for relevant merge_options
merge-recursive: rename MERGE_RECURSIVE_* to MERGE_VARIANT_*
merge-recursive: split internal fields into a separate struct
merge-recursive: avoid losing output and leaking memory holding that output
merge-recursive: comment and reorder the merge_options fields
merge-recursive: consolidate unnecessary fields in merge_options
merge-recursive: move some definitions around to clean up the header
merge-recursive: rename merge_options argument to opt in header
merge-recursive: rename 'mrtree' to 'result_tree', for clarity
merge-recursive: use common name for ancestors/common/base_list
merge-recursive: fix some overly long lines
cache-tree: share code between functions writing an index as a tree
merge-recursive: don't force external callers to do our logging
merge-recursive: remove useless parameter in merge_trees()
merge-recursive: exit early if index != head
Ensure index matches head before invoking merge machinery, round N
merge-recursive: remove another implicit dependency on the_repository
merge-recursive: future-proof update_file_flags() against memory leaks
merge-recursive: introduce an enum for detect_directory_renames values
...
Junio C Hamano [Wed, 18 Sep 2019 22:21:50 +0000 (15:21 -0700)]
Merge branch 'jh/trace2-pretty-output' into next
Output from trace2 subsystem is formatted more prettily now.
* jh/trace2-pretty-output:
trace2: cleanup whitespace in perf format
trace2: cleanup whitespace in normal format
quote: add sq_append_quote_argv_pretty()
trace2: trim trailing whitespace in normal format error message
trace2: remove dead code in maybe_add_string_va()
trace2: trim whitespace in region messages in perf target format
trace2: cleanup column alignment in perf target format
Junio C Hamano [Wed, 18 Sep 2019 22:21:50 +0000 (15:21 -0700)]
Merge branch 'dl/rebase-i-keep-base' into next
"git rebase --keep-base <upstream>" tries to find the original base
of the topic being rebased and rebase on top of that same base,
which is useful when running the "git rebase -i" (and its limited
variant "git rebase -x").
The command also has learned to fast-forward in more cases where it
can instead of replaying to recreate identical commits.
* dl/rebase-i-keep-base:
rebase: teach rebase --keep-base
rebase tests: test linear branch topology
rebase: fast-forward --fork-point in more cases
rebase: fast-forward --onto in more cases
rebase: refactor can_fast_forward into goto tower
t3432: test for --no-ff's interaction with fast-forward
t3432: distinguish "noop-same" v.s. "work-same" in "same head" tests
t3432: test rebase fast-forward behavior
t3431: add rebase --fork-point tests
Junio C Hamano [Wed, 18 Sep 2019 22:21:49 +0000 (15:21 -0700)]
Merge branch 'dl/complete-cherry-pick-revert-skip' into next
The command line completion support (in contrib/) learned about the
"--skip" option of "git revert" and "git cherry-pick".
* dl/complete-cherry-pick-revert-skip:
status: mention --skip for revert and cherry-pick
completion: add --skip for cherry-pick and revert
completion: merge options for cherry-pick and revert
Garima Singh [Mon, 26 Aug 2019 16:29:58 +0000 (09:29 -0700)]
commit-graph: add --[no-]progress to write and verify
Add --[no-]progress to git commit-graph write and verify.
The progress feature was introduced in 7b0f229
("commit-graph write: add progress output", 2018-09-17) but
the ability to opt-out was overlooked.
Signed-off-by: Garima Singh <garima.singh@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Stephen P. Smith [Thu, 12 Sep 2019 04:11:02 +0000 (21:11 -0700)]
test_date.c: remove reference to GIT_TEST_DATE_NOW
Remove the reference to the GIT_TEST_DATE_NOW which is done in date.c.
We can't get rid of the "x" variable, since it serves as a generic
scratch variable for parsing later in the function.
Signed-off-by: Stephen P. Smith <ischis2@cox.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Wed, 18 Sep 2019 18:50:09 +0000 (11:50 -0700)]
Merge branch 'md/list-objects-filter-combo'
The list-objects-filter API (used to create a sparse/lazy clone)
learned to take a combined filter specification.
* md/list-objects-filter-combo:
list-objects-filter-options: make parser void
list-objects-filter-options: clean up use of ALLOC_GROW
list-objects-filter-options: allow mult. --filter
strbuf: give URL-encoding API a char predicate fn
list-objects-filter-options: make filter_spec a string_list
list-objects-filter-options: move error check up
list-objects-filter: implement composite filters
list-objects-filter-options: always supply *errbuf
list-objects-filter: put omits set in filter struct
list-objects-filter: encapsulate filter components
Junio C Hamano [Wed, 18 Sep 2019 18:50:09 +0000 (11:50 -0700)]
Merge branch 'cc/multi-promisor'
Teach the lazy clone machinery that there can be more than one
promisor remote and consult them in order when downloading missing
objects on demand.
* cc/multi-promisor:
Move core_partial_clone_filter_default to promisor-remote.c
Move repository_format_partial_clone to promisor-remote.c
Remove fetch-object.{c,h} in favor of promisor-remote.{c,h}
remote: add promisor and partial clone config to the doc
partial-clone: add multiple remotes in the doc
t0410: test fetching from many promisor remotes
builtin/fetch: remove unique promisor remote limitation
promisor-remote: parse remote.*.partialclonefilter
Use promisor_remote_get_direct() and has_promisor_remote()
promisor-remote: use repository_format_partial_clone
promisor-remote: add promisor_remote_reinit()
promisor-remote: implement promisor_remote_get_direct()
Add initial support for many promisor remotes
fetch-object: make functions return an error code
t0410: remove pipes after git commands
Command line completion updates for "git -c var.name=val"
* sg/complete-configuration-variables:
completion: complete config variables and values for 'git clone --config='
completion: complete config variables names and values for 'git clone -c'
completion: complete values of configuration variables after 'git -c var='
completion: complete configuration sections and variable names for 'git -c'
completion: split _git_config()
completion: simplify inner 'case' pattern in __gitcomp()
completion: use 'sort -u' to deduplicate config variable names
completion: deduplicate configuration sections
completion: add tests for 'git config' completion
completion: complete more values of more 'color.*' configuration variables
completion: fix a typo in a comment
Junio C Hamano [Wed, 18 Sep 2019 18:50:07 +0000 (11:50 -0700)]
Merge branch 'jk/drop-release-pack-memory'
xmalloc() used to have a mechanism to ditch memory and address
space resources as the last resort upon seeing an allocation
failure from the underlying malloc(), which made the code complex
and thread-unsafe with dubious benefit, as major memory resource
users already do limit their uses with various other mechanisms.
It has been simplified away.
* jk/drop-release-pack-memory:
packfile: drop release_pack_memory()
Junio C Hamano [Wed, 18 Sep 2019 18:50:07 +0000 (11:50 -0700)]
Merge branch 'js/rebase-r-strategy'
"git rebase --rebase-merges" learned to drive different merge
strategies and pass strategy specific options to them.
* js/rebase-r-strategy:
t3427: accelerate this test by using fast-export and fast-import
rebase -r: do not (re-)generate root commits with `--root` *and* `--onto`
t3418: test `rebase -r` with merge strategies
t/lib-rebase: prepare for testing `git rebase --rebase-merges`
rebase -r: support merge strategies other than `recursive`
t3427: fix another incorrect assumption
t3427: accommodate for the `rebase --merge` backend having been replaced
t3427: fix erroneous assumption
t3427: condense the unnecessarily repetitive test cases into three
t3427: move the `filter-branch` invocation into the `setup` case
t3427: simplify the `setup` test case significantly
t3427: add a clarifying comment
rebase: fold git-rebase--common into the -p backend
sequencer: the `am` and `rebase--interactive` scripts are gone
.gitignore: there is no longer a built-in `git-rebase--interactive`
t3400: stop referring to the scripted rebase
Drop unused git-rebase--am.sh
René Scharfe [Wed, 18 Sep 2019 16:35:38 +0000 (18:35 +0200)]
sha1_name: simplify strbuf handling in interpret_nth_prior_checkout()
Pass the target strbuf to the callback function grab_nth_branch_switch()
by reference so that it can add the result string directly instead of
having it put the string into a temporary strbuf first. This gets rid
of an extra allocation and a string copy.
Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The shebang for a python script should be "/usr/bin/env python" and not
"/usr/bin/python". On some OSes like AIX, python default path is not under
"/usr/bin" ("/opt/freeware/bin" for AIX).
Note the main reason behind this change is that AIX rpm will add a
dependency on "/usr/bin/python" instead of "/usr/bin/env".
Signed-off-by: Clément Chigot <clement.chigot@atos.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano [Tue, 17 Sep 2019 22:07:52 +0000 (15:07 -0700)]
Merge branch 'master' into next
* master:
gitk: rename zh_CN.po to zh_cn.po
gitk: Do not mistake unchanged lines for submodule changes
gitk: Use right colour for remote refs in the "Tags and heads" dialog
gitk: Add Chinese (zh_CN) translation
gitk: Make web links clickable
Denton Liu [Tue, 17 Sep 2019 08:52:06 +0000 (01:52 -0700)]
gitk: rename zh_CN.po to zh_cn.po
When running make from a clean environment, all of the *.po files should
be converted into *.msg files. After that, when make is run without any
changes, make should not do anything.
After beffae768a (gitk: Add Chinese (zh_CN) translation, 2017-03-11),
zh_CN.po was introduced. When make was run, a zh_cn.msg file was
generated (notice the lowercase). However, since make is case-sensitive,
it expects zh_CN.po to generate a zh_CN.msg file so make will keep
reattempting to generate a zh_CN.msg so successive make invocations
result in
Denton Liu [Mon, 16 Sep 2019 19:23:16 +0000 (12:23 -0700)]
Makefile: run coccicheck on more source files
Before, when running the "coccicheck" target, only the source files
which were being compiled would have been checked by Coccinelle.
However, just because we aren't compiling a source file doesn't mean we
have to exclude it from analysis. This will allow us to catch more
mistakes, in particular ones that affect Windows-only sources since
Coccinelle currently runs only on Linux.
Make the "coccicheck" target run on all C sources except for those that
are taken from some third-party source. We don't want to patch these
files since we want them to be as close to upstream as possible so that
it'll be easier to pull in upstream updates.
When running a build on Arch Linux with no additional flags provided,
after applying this patch, the following sources are now checked:
This also results in the following source now being excluded:
* compat/obstack.c
Instead of generating $(FOUND_C_SOURCES) from a
`$(shell $(FIND_SOURCE_FILES))` invocation, an alternative design was
considered which involved converting $(FIND_SOURCE_FILES) into
$(SOURCE_FILES) which would hold a list of filenames from the
$(FIND_SOURCE_FILES) invocation. We would simply filter `%.c` files into
$(ALL_C_SOURCES). $(SOURCE_FILES) would then be passed directly to the
etags, ctags and cscope commands. We can see from the following
invocation
that the number of characters in this list would pose a problem on
platforms with short command-line length limits (such as CMD which has a
max of 8191 characters). As a result, we don't perform this change.
However, we can see that the same issue may apply when running
Coccinelle since $(COCCI_SOURCES) is also a list of filenames:
This is justified since platforms that support Coccinelle generally have
reasonably long command-line length limits and so we are safe for the
foreseeable future.
Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Denton Liu [Mon, 16 Sep 2019 19:23:14 +0000 (12:23 -0700)]
Makefile: strip leading ./ in $(FIND_SOURCE_FILES)
Currently, $(FIND_SOURCE_FILES) has two modes: if `git ls-files` is
present, it will use that to enumerate the files in the repository; else
it will use `$(FIND) .` to enumerate the files in the directory.
There is a subtle difference between these two methods, however. With
ls-files, filenames don't have a leading `./` while with $(FIND), they
do. This does not currently pose a problem but in a future patch, we
will be using `filter-out` to process the list of files with the
assumption that there is no prefix.
Unify the two possible invocations in $(FIND_SOURCE_FILES) by using sed
to remove the `./` prefix in the $(FIND) case.
Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Denton Liu [Mon, 16 Sep 2019 19:23:11 +0000 (12:23 -0700)]
Makefile: define THIRD_PARTY_SOURCES
Some files in our codebase are borrowed from other projects, and
minimally updated to suit our own needs. We'd sometimes need to tell
our own sources and these third-party sources apart for management
purposes (e.g. we may want to be less strict about coding style and
other issues on third-party files).
Define the $(MAKE) variable THIRD_PARTY_SOURCES that can be used to
match names of third-party sources.
Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
where I've elided a bunch of unnecessary details and PROCESS(&abs_path)
represents a big chunk of code rather than an actual function call. One
piece of PROCESS was:
if (lstat(abs_path.buf, &st))
continue;
which would cause the strbuf_reset() to be missed -- meaning that the
next path to be handled would have two paths concatenated. This path
used to use die_errno() instead of continue prior to commit 396049e5fb62
("git-clean: refactor git-clean into two phases", 2013-06-25), but my
understanding of how correct_untracked_entries() works is that it will
prevent both dir/ and dir/file from being in the list to clean so this
should be dead code and the die_errno() should be safe. But I hesitate
to remove it since I am not certain.
However, we can fix both this bug and possible similar future bugs by
simply moving the strbuf_reset(&abs_path) to the beginning of the loop.
It'll result in N calls to strbuf_reset() instead of N-1, but that's a
small price to pay to avoid sneaky bugs like this.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
clean: avoid removing untracked files in a nested git repository
Users expect files in a nested git repository to be left alone unless
sufficiently forced (with two -f's). Unfortunately, in certain
circumstances, git would delete both tracked (and possibly dirty) files
and untracked files within a nested repository. To explain how this
happens, let's contrast a couple cases. First, take the following
example setup (which assumes we are already within a git repo):
git init nested
cd nested
>tracked
git add tracked
git commit -m init
>untracked
cd ..
In this setup, everything works as expected; running 'git clean -fd'
will result in fill_directory() returning the following paths:
nested/
nested/tracked
nested/untracked
and then correct_untracked_entries() would notice this can be compressed
to
nested/
and then since "nested/" is a directory, we would call
remove_dirs("nested/", ...), which would
check is_nonbare_repository_dir() and then decide to skip it.
However, if someone also creates an ignored file:
>nested/ignored
then running 'git clean -fd' would result in fill_directory() returning
the same paths:
nested/
nested/tracked
nested/untracked
but correct_untracked_entries() will notice that we had ignored entries
under nested/ and thus simplify this list to
nested/tracked
nested/untracked
Since these are not directories, we do not call remove_dirs() which was
the only place that had the is_nonbare_repository_dir() safety check --
resulting in us deleting both the untracked file and the tracked (and
possibly dirty) file.
One possible fix for this issue would be walking the parent directories
of each path and checking if they represent nonbare repositories, but
that would be wasteful. Even if we added caching of some sort, it's
still a waste because we should have been able to check that "nested/"
represented a nonbare repository before even descending into it in the
first place. Add a DIR_SKIP_NESTED_GIT flag to dir_struct.flags and use
it to prevent fill_directory() and friends from descending into nested
git repos.
With this change, we also modify two regression tests added in commit 91479b9c72f1 ("t7300: add tests to document behavior of clean and nested
git", 2015-06-15). That commit, nor its series, nor the six previous
iterations of that series on the mailing list discussed why those tests
coded the expectation they did. In fact, it appears their purpose was
simply to test _existing_ behavior to make sure that the performance
changes didn't change the behavior. However, these two tests directly
contradicted the manpage's claims that two -f's were required to delete
files/directories under a nested git repository. While one could argue
that the user gave an explicit path which matched files/directories that
were within a nested repository, there's a slippery slope that becomes
very difficult for users to understand once you go down that route (e.g.
what if they specified "git clean -f -d '*.c'"?) It would also be hard
to explain what the exact behavior was; avoid such problems by making it
really simple.
Also, clean up some grammar errors describing this functionality in the
git-clean manpage.
Finally, there are still a couple bugs with -ffd not cleaning out enough
(e.g. missing the nested .git) and with -ffdX possibly cleaning out the
wrong files (paying attention to outer .gitignore instead of inner).
This patch does not address these cases at all (and does not change the
behavior relative to those flags), it only fixes the handling when given
a single -f. See
https://public-inbox.org/git/20190905212043.GC32087@szeder.dev/ for more
discussion of the -ffd[X?] bugs.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
The -d flag pre-dated git-clean's ability to have paths specified. As
such, the default for git-clean was to only remove untracked files in
the current directory, and -d existed to allow it to recurse into
subdirectories.
The interaction of paths and the -d option appears to not have been
carefully considered, as evidenced by numerous bugs and a dearth of
tests covering such pairings in the testsuite. The definition turns out
to be important, so let's look at some of the various ways one could
interpret the -d option:
A) Without -d, only look in subdirectories which contain tracked
files under them; with -d, also look in subdirectories which
are untracked for files to clean.
B) Without specified paths from the user for us to delete, we need to
have some kind of default, so...without -d, only look in
subdirectories which contain tracked files under them; with -d,
also look in subdirectories which are untracked for files to clean.
The important distinction here is that choice B says that the presence
or absence of '-d' is irrelevant if paths are specified. The logic
behind option B is that if a user explicitly asked us to clean a
specified pathspec, then we should clean anything that matches that
pathspec. Some examples may clarify. Should
git clean -f untracked_dir/file
remove untracked_dir/file or not? It seems crazy not to, but a strict
reading of option A says it shouldn't be removed. How about
? Should it remove either or both of these files? Should it require
multiple runs to remove both the files listed? (If this sounds like a
crazy question to even ask, see the commit message of "t7300: Add some
testcases showing failure to clean specified pathspecs" added earlier in
this patch series.) What if -ffd were used instead of -f -- should that
allow these to be removed? Should it take multiple invocations with
-ffd? What if a glob (such as '*tracked*') were used instead of
spelling out the directory names? What if the filenames involved globs,
such as
git clean -f '*.o'
or
git clean -f '*/*.o'
?
The current documentation actually suggests a definition that is
slightly different than choice A, and the implementation prior to this
series provided something radically different than either choices A or
B. (The implementation, though, was clearly just buggy). There may be
other choices as well. However, for almost any given choice of
definition for -d that I can think of, some of the examples above will
appear buggy to the user. The only case that doesn't have negative
surprises is choice B: treat a user-specified path as a request to clean
all untracked files which match that path specification, including
recursing into any untracked directories.
Change the documentation and basic implementation to use this
definition.
There were two regression tests that indirectly depended on the current
implementation, but neither was about subdirectory handling. These two
tests were introduced in commit 5b7570cfb41c ("git-clean: add tests for
relative path", 2008-03-07) which was solely created to add coverage for
the changes in commit fb328947c8e ("git-clean: correct printing relative
path", 2008-03-07). Both tests specified a directory that happened to
have an untracked subdirectory, but both were only checking that the
resulting printout of a file that was removed was shown with a relative
path. Update these tests appropriately.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir: add commentary explaining match_pathspec_item's return value
The way match_pathspec_item() handles names and pathspecs with trailing
slash characters, in conjunction with special options like
DO_MATCH_DIRECTORY and DO_MATCH_LEADING_PATHSPEC were non-obvious, and
broken until this patch series. Add a table in a comment explaining the
intent of how these work.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir: if our pathspec might match files under a dir, recurse into it
For git clean, if a directory is entirely untracked and the user did not
specify -d (corresponding to DIR_SHOW_IGNORED_TOO), then we usually do
not want to remove that directory and thus do not recurse into it.
However, if the user manually specified specific (or even globbed) paths
somewhere under that directory to remove, then we need to recurse into
the directory to make sure we remove the relevant paths under that
directory as the user requested.
Note that this does not mean that the recursed-into directory will be
added to dir->entries for later removal; as of a few commits earlier in
this series, there is another more strict match check that is run after
returning from a recursed-into directory before deciding to add it to the
list of entries. Therefore, this will only result in files underneath
the given directory which match one of the pathspecs being added to the
entries list.
Two notes of potential interest to future readers:
* If we wanted to only recurse into a directory when it is specifically
matched rather than matched-via-glob (e.g. '*.c'), then we could do
so via making the final non-zero return in match_pathspec_item be
MATCHED_RECURSIVELY instead of MATCHED_RECURSIVELY_LEADING_PATHSPEC.
(Note that the relative order of MATCHED_RECURSIVELY_LEADING_PATHSPEC
and MATCHED_RECURSIVELY are important for such a change.) I was
leaving open that possibility while writing an RFC asking for the
behavior we want, but even though we don't want it, that knowledge
might help you understand the code flow better.
* There is a growing amount of logic in read_directory_recursive() for
deciding whether to recurse into a subdirectory. However, there is a
comment immediately preceding this logic that says to recurse if
instructed by treat_path(). It may be better for the logic in
read_directory_recursive() to ultimately be moved to treat_path() (or
another function it calls, such as treat_directory()), but I have
left that for someone else to tackle in the future.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir: make the DO_MATCH_SUBMODULE code reusable for a non-submodule case
The specific checks done in match_pathspec_item for the DO_MATCH_SUBMODULE
case are useful for other cases which have nothing to do with submodules.
Rename this constant; a subsequent commit will make use of this change.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
dir: also check directories for matching pathspecs
Even if a directory doesn't match a pathspec, it is possible, depending
on the precise pathspecs, that some file underneath it might. So we
special case and recurse into the directory for such situations. However,
we previously always added any untracked directory that we recursed into
to the list of untracked paths, regardless of whether the directory
itself matched the pathspec.
For the case of git-clean and a set of pathspecs of "dir/file" and "more",
this caused a problem because we'd end up with dir entries for both of
"dir"
"dir/file"
Then correct_untracked_entries() would try to helpfully prune duplicates
for us by removing "dir/file" since it's under "dir", leaving us with
"dir"
Since the original pathspec only had "dir/file", the only entry left
doesn't match and leaves nothing to be removed. (Note that if only one
pathspec was specified, e.g. only "dir/file", then the common_prefix_len
optimizations in fill_directory would cause us to bypass this problem,
making it appear in simple tests that we could correctly remove manually
specified pathspecs.)
Fix this by actually checking whether the directory we are about to add
to the list of dir entries actually matches the pathspec; only do this
matching check after we have already returned from recursing into the
directory.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
For a pathspec like 'foo/bar' comparing against a path named "foo/",
namelen will be 4, and match[namelen] will be 'b'. The correct location
of the directory separator is namelen-1.
However, other callers of match_pathspec_item() such as builtin/grep.c's
submodule_path_match() will compare against a path named "foo" instead of
"foo/". It might be better to change all the callers to be consistent,
as discussed at
https://public-inbox.org/git/xmqq7e6cdnkr.fsf@gitster-ct.c.googlers.com/
and
https://public-inbox.org/git/CABPp-BERWUPCPq-9fVW1LNocqkrfsoF4BPj3gJd9+En43vEkTQ@mail.gmail.com/
but there are many cases to audit, so for now just make sure we handle
both cases with and without a trailing slash.
The reason the code worked despite this sometimes-off-by-one error was
that the subsequent code immediately checked whether the first matchlen
characters matched (which they do) and then bailed and return
MATCHED_RECURSIVELY anyway since wildmatch doesn't have the ability to
check if "name" can be matched as a directory (or prefix) against the
pathspec.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
t7300: add testcases showing failure to clean specified pathspecs
Someone brought me a testcase where multiple git-clean invocations were
required to clean out unwanted files:
mkdir d{1,2}
touch d{1,2}/ut
touch d1/t && git add d1/t
With this setup, the user would need to run
git clean -ffd */ut
twice to delete both ut files.
A little testing showed some interesting variants:
* If only one of those two ut files existed (either one), then only one
clean command would be necessary.
* If both directories had tracked files, then only one git clean would
be necessary to clean both files.
* If both directories had no tracked files then the clean command above
would never clean either of the untracked files despite the pathspec
explicitly calling both of them out.
A bisect showed that the failure to clean out the files started with
commit cf424f5fd89b ("clean: respect pathspecs with "-d", 2014-03-10).
However, that pointed to a separate issue: while the "-d" flag was used
by the original user who showed me this problem, that flag should have
been irrelevant to this problem. Testing again without the "-d" flag
showed that the same buggy behavior exists without using that flag, and
has in fact existed since before cf424f5fd89b.
Although these problems at first are perceived to be different (e.g.
never clearing out the requested files vs. taking multiple invocations
to get everything cleared out), they are actually just different
manifestations of the same problem. The case with multiple directories
that have no tracked files is the more general case; solving it will
solve all the others. So, I concentrate on it. Add testcases showing
that multiple untracked files within entirely untracked directories
cannot be cleaned when specifying these files to git clean via
pathspecs.
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Johannes Sixt [Mon, 16 Sep 2019 20:46:21 +0000 (22:46 +0200)]
diff, log doc: say "patch text" instead of "patches"
diff, log doc: say "patch text" instead of "patches"
A poster on Stackoverflow was confused that the documentation of git-log
promised to generate "patches" or "patch files" with -p, but there were
none to be found. Rewrite the corresponding paragraph to talk about
"patch text" to avoid the confusion.
Shorten the language to say "X does Y" in place of "X does not Z, but Y".
Cross-reference the referred-to commands like the rest of the file does.
Enumerate git-show because it includes the description as well.
Mention porcelain commands before plumbing commands because I guess that
the paragraph is read more frequently in their context.
Signed-off-by: Johannes Sixt <j6t@kdbg.org> Acked-by: Martin Ågren <martin.agren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor [Mon, 16 Sep 2019 20:54:12 +0000 (22:54 +0200)]
Test the progress display
'progress.c' has seen a few fixes recently [1], and, unfortunately,
some of those fixes required further fixes [2]. It seems it's time to
have a few tests focusing on the subtleties of the progress display.
Add the 'test-tool progress' subcommand to help testing the progress
display, reading instructions from standard input and turning them
into calls to the display_progress() and display_throughput()
functions with the given parameters.
The progress display is, however, critically dependent on timing,
because it's only updated once every second or, if the toal is known
in advance, every 1%, and there is the throughput rate as well. These
make the progress display far too undeterministic for testing as-is.
To address this, add a few testing-specific variables and functions to
'progress.c', allowing the the new test helper to:
- Disable the triggered-every-second SIGALRM and set the
'progress_update' flag explicitly based in the input instructions.
This way the progress line will be updated deterministically when
the test wants it to be updated.
- Specify the time elapsed since start_progress() to make the
throughput rate calculations deterministic.
Add the new test script 't0500-progress-display.sh' to check a few
simple cases with and without throughput, and that a shorter progress
line properly covers up the previously displayed line in different
situations.
[1] See commits 545dc345eb (progress: break too long progress bar
lines, 2019-04-12) and 9f1fd84e15 (progress: clear previous
progress update dynamically, 2019-04-12).
[2] 1aed1a5f25 (progress: avoid empty line when breaking the progress
line, 2019-05-19)
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
SZEDER Gábor [Mon, 16 Sep 2019 20:54:11 +0000 (22:54 +0200)]
Revert "progress: use term_clear_line()"
This reverts commit 5b12e3123b (progress: use term_clear_line(),
2019-06-24), because covering up the entire last line while refreshing
the progress line caused unexpected problems during 'git
clone/fetch/push':
The length of the progress bar line can shorten when it includes
throughput and the unit changes, or when its length exceeds the width
of the terminal and is broken into two lines. In these cases the
previously displayed longer progress line should be covered up,
because otherwise the leftover characters from the previous progress
line make the output look weird [1]. term_clear_line() makes this
quite simple, as it covers up the entire last line either by using an
ANSI control sequence or by printing a terminal width worth of space
characters, depending on whether the terminal is smart or dumb.
Unfortunately, when accessing a remote repository via any non-local
protocol the remote 'git receive-pack/upload-pack' processes can't
possibly have any idea about the local terminal (smart of dumb? how
wide?) their progress will end up on. Consequently, they assume the
worst, i.e. standard-width dumb terminal, and print 80 spaces to cover
up the previously displayed progress line. The local 'git
clone/fetch/push' processes then display the remote's progress,
including these coverup spaces, with the 'remote: ' prefix, resulting
in a total line length of 88 characters. If the local terminal is
narrower than that, then the coverup gets line-wrapped, and after that
the CR at the end doesn't return to the beginning of the progress
line, but to the first column of its last line, resulting in those
repeated 'remote: <many-spaces>' lines.
By reverting 5b12e3123b (progress: use term_clear_line(),
2019-06-24) we won't cover up the entire last line, but go back to
comparing the length of the current progress bar line with the
previous one, and cover up as many characters as needed.
[1] See commits 545dc345eb (progress: break too long progress bar
lines, 2019-04-12) and 9f1fd84e15 (progress: clear previous
progress update dynamically, 2019-04-12).
Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Denton Liu [Mon, 16 Sep 2019 19:23:08 +0000 (12:23 -0700)]
Makefile: strip leading ./ in $(LIB_H)
Currently, $(LIB_H) is generated from two modes: if `git ls-files` is
present, it will use that to enumerate the files in the repository; else
it will use `$(FIND) .` to enumerate the files in the directory.
There is a subtle difference between these two methods, however. With
ls-files, filenames don't have a leading `./` while with $(FIND), they
do. This results in $(CHK_HDRS) having to substitute out the leading
`./` before it uses $(LIB_H).
Unify the two possible values in $(LIB_H) by using patsubst to remove the
`./` prefix at its definition.
Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
fetch: use oidset to keep the want OIDs for faster lookup
During git-fetch, the client checks if the advertised tags' OIDs are
already in the fetch request's want OID set. This check is done in a
linear scan. For a repository that has a lot of refs, repeating this
scan takes 15+ minutes. In order to speed this up, create a oid_set for
other refs' OIDs.
Signed-off-by: Masaya Suzuki <masayasuzuki@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Sun, 15 Sep 2019 17:07:44 +0000 (19:07 +0200)]
commit-graph: use commit_list_count()
Let commit_list_count() count the number of parents instead of
duplicating it. Also store the result in an unsigned int, as that's
what the function returns, and the count is never negative.
Signed-off-by: René Scharfe <l.s.r@web.de> Acked-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Sun, 15 Sep 2019 12:10:28 +0000 (14:10 +0200)]
sha1-name: check for overflow of N in "foo^N" and "foo~N"
Reject values that don't fit into an int, as get_parent() and
get_nth_ancestor() cannot handle them. That's better than potentially
returning a random object.
If this restriction turns out to be too tight then we can switch to a
wider data type, but we'd still have to check for overflow.
Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
René Scharfe [Sun, 15 Sep 2019 12:03:25 +0000 (14:03 +0200)]
rev-parse: demonstrate overflow of N for "foo^N" and "foo~N"
If the number gets too high for an int, weird things may happen, as
signed overflows are undefined. Add a test to show this; rev-parse
"sucessfully" interprets 100000000000000000000000000000000 to be the
same as 0, at least on x64 with GCC 9.2.1 and Clang 8.0.1, which is
obviously bogus.
Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Sun, 15 Sep 2019 16:51:56 +0000 (12:51 -0400)]
list-objects-filter: use empty string instead of NULL for sparse "base"
We use add_excludes_from_blob_to_list() to parse a sparse blob. Since
we don't have a base path, we pass NULL and 0 for the base and baselen,
respectively. But the rest of the exclude code passes a literal empty
string instead of NULL for this case. And indeed, we eventually end up
with match_pathname() calling fspathncmp(), which then calls the system
strncmp(path, base, baselen).
This works on many platforms, which notice that baselen is 0 and do not
look at the bytes of "base" at all. But it does violate the C standard,
and building with SANITIZE=undefined will complain. You can also see it
by instrumenting fspathncmp like this:
Jon Simons [Sun, 15 Sep 2019 01:13:47 +0000 (21:13 -0400)]
list-objects-filter: give a more specific error sparse parsing error
The sparse:oid filter has two error modes: we might fail to resolve the
name to an OID, or we might fail to parse the contents of that OID. In
the latter case, let's give a less generic error message, and mention
the OID we did find.
While we're here, let's also mark both messages as translatable.
Signed-off-by: Jeff King <peff@peff.net> Acked-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
Jeff King [Sun, 15 Sep 2019 16:12:44 +0000 (12:12 -0400)]
list-objects-filter: delay parsing of sparse oid
The list-objects-filter code has two steps to its initialization:
1. parse_list_objects_filter() makes sure the spec is a filter we know
about and is syntactically correct. This step is done by "rev-list"
or "upload-pack" that is going to apply a filter, but also by "git
clone" or "git fetch" before they send the spec across the wire.
2. list_objects_filter__init() runs the type-specific initialization
(using function pointers established in step 1). This happens at
the start of traverse_commit_list_filtered(), when we're about to
actually use the filter.
It's a good idea to parse as much as we can in step 1, in order to catch
problems early (e.g., a blob size limit that isn't a number). But one
thing we _shouldn't_ do is resolve any oids at that step (e.g., for
sparse-file contents specified by oid). In the case of a fetch, the oid
has to be resolved on the remote side.
The current code does resolve the oid during the parse phase, but
ignores any error (which we must do, because we might just be sending
the spec across the wire). This leads to two bugs:
- if we're not in a repository (e.g., because it's git-clone parsing
the spec), then we trigger a BUG() trying to resolve the name
- if we did hit the error case, we still have to notice that later and
bail. The code path in rev-list handles this, but the one in
upload-pack does not, leading to a segfault.
We can fix both by moving the oid resolution into the sparse-oid init
function. At that point we know we have a repository (because we're
about to traverse), and handling the error there fixes the segfault.
As a bonus, we can drop the NULL sparse_oid_value check in rev-list,
since this is now handled in the sparse-oid-filter init function.
Signed-off-by: Jeff King <peff@peff.net> Acked-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
projects / git / log