Stefan Tatschner [Mon, 13 Jan 2014 21:10:45 +0000 (22:10 +0100)]
filters: Improved syntax-highlighting.py
- Switched back to python2 according to a problem in pygments with python3.
With the next release of pygments this problem should be fixed.
Issue see here:
https://bitbucket.org/birkenfeld/pygments-main/issue/901/problems-with-python3
- Just read the stdin, decode it to utf-8 and ignore unknown signs. This ensures
that even destroyed files do not cause any errors in the filter.
- Improved language guessing:
-> At first use guess_lexer_for_filename for a better detection of the used
programming languages (even mixed cases will be detected, e.g. php + html).
-> If nothing was found look if there is a shebang and use guess_lexer.
-> As default/fallback choose TextLexer.
Signed-off-by: Stefan Tatschner <stefan@sevenbyte.org>
John Keeping [Sun, 12 Jan 2014 19:45:16 +0000 (19:45 +0000)]
ui-shared: URL-escape script_name
As far as I know, there is no requirement that $SCRIPT_NAME contain only
URL-safe characters, so we need to make sure that any special characters
are escaped.
John Keeping [Sun, 12 Jan 2014 17:13:50 +0000 (17:13 +0000)]
filter: pass extra arguments via cgit_open_filter
This avoids poking into the filter data structure at various points in
the code. We rely on the fact that the number of arguments is fixed
based on the filter type (set in cgit_new_filter) and that the call
sites all know which filter type they're using.
John Keeping [Sun, 12 Jan 2014 17:13:49 +0000 (17:13 +0000)]
ui-snapshot: set unused cgit_filter fields to zero
By switching the assignment of fields in the cgit_filter structure to
use designated initializers, the compiler will initialize all other
fields to their default value. This will be needed when we add the
extra_args field in the next patch.
==18344== Conditional jump or move depends on uninitialised value(s)
==18344== at 0x406C83: open_slot (cache.c:63)
==18344== by 0x407478: cache_ls (cache.c:403)
==18344== by 0x404C9A: process_request (cgit.c:639)
==18344== by 0x406BD2: fill_slot (cache.c:190)
==18344== by 0x4071A0: cache_process (cache.c:284)
==18344== by 0x404461: main (cgit.c:952)
==18344== Uninitialised value was created by a stack allocation
==18344== at 0x40738B: cache_ls (cache.c:375)
This is caused by the keylen field being used to calculate whether or
not a slot is matched. We never then check the value of this and the
length of data read depends on the key length read from the file so this
isn't dangerous, but it's nice to avoid branching based on uninitialized
data.
It's only used in one place, and not useful to have around since
close_filter will die() if exit_status isn't what it expects, anyway. So
this is best as just a local variable instead of as part of the struct.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Lukas Fleischer [Fri, 10 Jan 2014 13:55:30 +0000 (14:55 +0100)]
cgit.c: Use "else" for mutually exclusive branches
When parsing command line arguments, no pair of command line options can
ever match simultaneously. Use "else if" blocks to reflect this. This
change improves both readability and speed.
Lukas Fleischer [Fri, 10 Jan 2014 11:44:36 +0000 (12:44 +0100)]
Disallow use of undocumented snapshot delimiters
Since the introduction of selective snapshot format configuration in dc3c9b5 (allow selective enabling of snapshots, 2007-07-21), we allowed
seven different delimiters for snapshot formats, while the documentation
has always been clear about spaces being the only valid delimiter:
The value is a space-separated list of zero or more of the values
"tar", "tar.gz", "tar.bz2", "tar.xz" and "zip".
Supporting the undocumented delimiters makes the code unnecessarily
complex. Remove them.
John Keeping [Sun, 6 Oct 2013 11:14:41 +0000 (12:14 +0100)]
plain: don't append charset for binary MIME types
When outputting the Content-Type HTTP header we print the MIME type and
then append "; charset=<charset>" if the charset variable is non-null.
We don't want a charset when we have selected "application/octet-stream"
or when the user has specified a custom MIME type, since they may have
specified their own charset. To avoid this, make sure we set the page's
charset to NULL in ui-plain before we generate the HTTP headers.
Signed-off-by: John Keeping <john@keeping.me.uk> Signed-off-by: Lukas Fleischer <cgit@cryptocrack.de>
Lukas Fleischer [Tue, 27 Aug 2013 08:40:51 +0000 (10:40 +0200)]
ui-shared: Drop filepair_cb_raw() and helper
Remove filepair_cb_raw() and all related functions. These are no longer
needed. We now use Git's internal functions for raw diff formatting
everywhere.
Lukas Fleischer [Tue, 20 Aug 2013 16:56:13 +0000 (18:56 +0200)]
ui-patch.c: Use log_tree_commit() to generate diffs
Instead of using our own formatting, use log_tree_commit() from Git to
create patches. This removes unnecessary duplicate code and also fixes a
bug with e-mail address formatting that existed in our own
implementation.
Lukas Fleischer [Tue, 20 Aug 2013 16:56:12 +0000 (18:56 +0200)]
ui-diff: Check the return value of get_sha1()
Sync with what we do everywhere else and check the return value of
get_sha1() instead of calling sha1_object_info() to validate the object.
Note that we later call lookup_commit_reference(), which checks that
both SHA1 values refer to commits, anyway.
Lukas Fleischer [Fri, 28 Jun 2013 08:58:14 +0000 (08:58 +0000)]
Fix section-from-path > 1
When having found the first path separator occurrence at position i, we
invoked strchr() on the same position i in subsequent iterations
resulting in the same path separator being returned by strchr() over and
over again. Increase the position by one to skip the occurrence that has
just been found and advance to the next separator.
Reported-by: Konstantin Ryabitsev <mricon@kernel.org> Signed-off-by: Lukas Fleischer <cgit@cryptocrack.de>
Lukas Fleischer [Tue, 4 Jun 2013 14:47:53 +0000 (14:47 +0000)]
Use strbuf for reading configuration files
Use struct strbuf from Git instead of fixed-size buffers to remove the
limit on the length of configuration file lines and refactor
read_config_line() to improve readability.
Note that this also fixes a buffer overflow that existed with the
original fixed-size buffer implementation.
We've long supported negative ttls, for infinite cache, except the
documentation incorrectly showed one of our defaults as being 5 and not
-1. As well, with a negative ttl, we were actually making the HTTP
expired header go backwards. This changes it to go ahead ten years
instead.
Further, we add an cache-about-ttl option to set a different ttl for
about pages, which are now increasingly being filtered through markdown
or just sent statically anyway.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
My dmesg is filled with the oom killer bringing down processes while the
Bingbot downloads every snapshot for every commit of the Linux kernel in
tar.xz format. Sure, I should be running with memory limits, and now I'm
using cgroups, but a more general solution is to prevent crawlers from
wasting resources like that in the first place.
Suggested-by: Natanael Copa <ncopa@alpinelinux.org> Suggested-by: Julius Plenz <plenz@cis.fu-berlin.de> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Features:
- update to git v1.8.3.
- expanded set of default filters to include markdown, restructuredtext, and
man pages.
- better sample configuration file in man page.
- "readme" may now be specified multiple times, and cgit will choose the first
one it finds.
- "readme" no longer needs a branch name. If prefixed with simply ":" it will
use the default branch.
- "branch-sort" allowing branches to be sorted either by "age" or "name", for
kernel.org.
- "enable-index-owner" allowing the owner column to be disabled in the index
page.
- print submodule revision next to submodule link.
- integrate more closely with git apis, such as strbuf.
- rely on git test harness and git makefiles.
- more robust test suite.
- more rebust makefile dependency accounting.
- pager navigation is now unordered list.
- span tag wraps commit directions.
Behavior changes:
- HOME is no longer passed as an environment variable to any filter api
scripts.
- "about-filter" now receives the filename being filtered as argv[1]. This may
disrupt existing scripts, so adjust accordingly.
- gitconfig and gitattributes are no longer loaded from any system directories
or home directories.
Security:
- CVE-2013-2117: disallow directory traversal when readme is set to filesystem
path.
Bug fixes:
- ssdiff now correctly manages tab expansion.
- support unannotated tags in http git clone.
- lots of cleanups of global variables and memory leaks.
- do not rely on gettext/libintl.
- better C standard compliance.
- make several functions and variables static.
- improved constification.
- remove unused functions.
- fix colspan values to correct width.
- fix out-of-bounds memory accesses with virtual_root="".
- cache repo config more precisely.
- die when write fails.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Using the url= query string, it was possible request arbitrary files
from the filesystem if the readme for a given page was set to a
filesystem file. The following request would return my /etc/passwd file:
This fix uses realpath(3) to canonicalize all paths, and then compares
the base components.
This fix introduces a subtle timing attack, whereby a client can check
whether or not strstr is called using timing measurements in order
to determine if a given file exists on the filesystem.
This fix also does not account for filesystem race conditions (TOCTOU)
in resolving symlinks.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
The readme variable may now contain multiple space deliminated entries,
which per usual are either a filepath or a git ref filepath. If multiple
are specified, cgit will now select the first one in the list that
exists. This is to make it easier to specify multiple default readme
types in the main cgitrc file and have them automatically get applied to
each repo based on what exists.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
John Keeping [Sat, 18 May 2013 17:46:39 +0000 (18:46 +0100)]
cache.c: cache ls_cache output properly
By using the standard library's printf, cache_ls does not redirect its
output to the cache when we change the process' stdout file descriptor
to point to the cache file. Fix this by using "htmlf" in the same way
that we do for writing HTTP headers.
John Keeping [Sat, 18 May 2013 17:46:38 +0000 (18:46 +0100)]
tests: introduce strip_header() helper function
This means that we can avoid hardcoding the number of headers we expect
CGit to generate in test cases and simply remove whatever headers happen
to by there when we are checking body content.
John Keeping [Sat, 18 May 2013 14:57:03 +0000 (15:57 +0100)]
html.c: die when write fails
If we fail to write HTML output once, there's no point carrying on so
just write a failure message once and die. By using Git's die_errno
function we also let the user know in what way the write failed.
John Keeping [Sat, 18 May 2013 14:54:49 +0000 (15:54 +0100)]
ui-log: add <span/> around commit decorations
This helps projects that have a large number of tags to display them all
using custom CSS.
The default stylesheet has not been updated since what is useful for
projects with a lot of tags is not the same as what is useful for
projects with only a small number of decorations per commit.
Suggested-by: Konstantin Ryabitsev <mricon@kernel.org> Signed-off-by: John Keeping <john@keeping.me.uk>
John Keeping [Sat, 18 May 2013 14:17:08 +0000 (15:17 +0100)]
Makefile: fix parallel "make test"
When building the "test" target we depend on both cgit and building the
Git tools. By doing this with two targets we end up running make in the
git/ directory twice, concurrently if using parallel make, which causes
us to build more than we need and potentially builds incorrectly if
multi-step build-then-move operations overlap.
Fix this by instead calling back into the makefile so that we alter the
"cgit" target to also build the Git tools.
John Keeping [Sat, 18 May 2013 17:28:14 +0000 (18:28 +0100)]
cache.c: fix cache_ls
Commit fb3655d (use struct strbuf instead of static buffers, 2013-04-06)
broke the logic in cache.c::cache_ls by failing to set slot->cache_name
before calling open_slot.
While fixing this, also free the strbufs added by that commit once we're
done with them.
It's a bit tedious to have to do this here too. If we encounter other
issues with $HOME down the line, I'll look into adding some nice utility
functions to handle this, or perhaps giving up on the hope that we could
keep $HOME defined for scripts.
This commit additionally adds a test case, should the issue surface
again.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
John Keeping [Sat, 27 Apr 2013 09:48:56 +0000 (10:48 +0100)]
t0001: validate Git -rcN version numbers correctly
When creating the GIT-VERSION-FILE that we use to test that the version
of Git in git/ is the same as in the CGit Makefile, Git applies the
transform "s/-/./g" to the version string. This doesn't affect released
versions but does change RC version numbers such as 1.8.3-rc0.
While CGit should only refer to a released Git version in general, it is
useful to developers who want to test upcoming Git releases if the tests
do work with RCs, so change t0001 to apply the same transform to our
Makefile version before comparing it to the contents of
GIT-VERSION-FILE.
John Keeping [Tue, 16 Apr 2013 17:39:58 +0000 (18:39 +0100)]
scan-tree: fix regression in section-from-path=-1
Commit fb3655d (use struct strbuf instead of static buffers -
2013-04-06) introduced a regression in the "section-from-path" handling
when the configured value is negative. By changing the "rel" variable
so that it includes a trailing slash, counting slashes from the end of
the string no longer gives the same answer as it did before.
Fix this by ensuring that "rel" does not have a trailing slash.
Reported-by: Julius Plenz <plenz@cis.fu-berlin.de> Signed-off-by: John Keeping <john@keeping.me.uk>
John Keeping [Sun, 14 Apr 2013 18:15:43 +0000 (19:15 +0100)]
t0001: ignore ".dirty" suffix on Git version
When testing modifications in Git that affect CGit, it is annoying to
have t0001 failing simply because the Git version has a ".dirty" suffix
when the version of Git there does indeed match that specified in the
CGit makefile. Stop this by stripping the ".dirty" suffix from the
GIT_VERSION variable.
Note that this brings the "Git version" behaviour in line with the
"submodule version" case which does not check if the working tree in
git/ is modified.
John Keeping [Sun, 14 Apr 2013 16:59:30 +0000 (17:59 +0100)]
tests: set TEST_OUTPUT_DIRECTORY to the CGit test directory
By default, Git's test suite puts the trash directories and test-results
directory into its own directory, not that containing the tests being
run. This is less convenient for inspecting test failures, so set the
output directory to CGit's tests/ directory instead.
Note that there is currently a bug in Git whereby it will create the
trash directories in our tests/ directory regardless of the value of
TEST_OUTPUT_DIRECTORY, and then fail to remove them once the tests are
done. This change does currently affect the location of the
test-results/ directory though.
John Keeping [Sun, 14 Apr 2013 16:07:41 +0000 (17:07 +0100)]
t0109: test more URLs
In order to ensure that we don't access $HOME at some point after
initial startup when rendering a specific view, run the strace test on a
range of different pages.
This ensures that we don't end up reading a configuration later for some
specific view.
ui-snapshot.c: Prepend "V" when guessing ref names
In cgit_print_snapshot_links() we strip leading "v" and "V", while we
currently only prepend a lower case "v" when parsing a snapshot file
name. This results in broken snapshot links for tags that start with an
upper case "V". Avoid this by prepending a "V" as a fallback.
Note that we cannot use skip_all here since some tests have already been
executed when ZIP tests are reached. Use test prerequisites to skip
everything using unzip(1) if the binary is not available instead.
John Keeping [Wed, 10 Apr 2013 12:11:57 +0000 (13:11 +0100)]
t0109: chain operations with &&
Without '&&' between operations, we will not detect if strace or cgit
exit with an error status, which would cause a false positive test
status in this case.
projects / cgit / log