John Koleszar [Tue, 26 Oct 2010 20:22:22 +0000 (16:22 -0400)]
vpxenc: add deterministic output option
By baking the version number into the output file, a hash of the file
will vary from commit to commit, even if the output is otherwise bit
exact. Add a -D option to suppress this behavior, for use when
bisecting or other debugging.
John Koleszar [Fri, 22 Oct 2010 03:40:42 +0000 (20:40 -0700)]
vpxdec: rework default output parameters
This patch reworks the default behavior of the tool to output Y4M
instead of writing individual raw frames. The relevant controls are
now:
--yv12, --i420 - These options change the output format to be
raw planar data. The output will be Y4M unless
one of these options is specified.
--flipuv - Swaps the chroma planes. Works with Y4M output.
-o, --output - Sets the output filename. Defaults to stdout if
not specified. Supports escape character
expansion for frame width (%w) height (%h) and
sequence number (%1..%9). The --prefix option
has been removed in favor of this escape
expansion.
Since the output defaults to stdout if -o is not specified, an
error will be thrown if stdout is not connected to a pipe. This
can be overridden by specifying '-o -'.
John Koleszar [Thu, 21 Oct 2010 20:53:52 +0000 (16:53 -0400)]
usage: fix horizontal alignment of options
When showing the command usage information for vpxenc and vpxdec,
options with both a short and long version that do not take an
argument were not properly aligned.
John Koleszar [Thu, 21 Oct 2010 19:02:10 +0000 (15:02 -0400)]
rename ivf{enc,dec} to vpx{enc,dec}
The new WebM output support should be preferred to IVF, but we can't
change the default behavior of the ivf* tools. There are a few other
default behaviors for these tools that are counterintuitive for
historical reasons, and changing the binary name provides the
opportunity to clean those up as well. This patch takes the first
step by renaming the binaries.
Frank Galligan [Wed, 6 Oct 2010 16:51:00 +0000 (12:51 -0400)]
Fixed the timebase parameter of ivfenc.
Ivfenc will use timebase if it is set. If it is not set ivfenc will
still double the timebase so altref frames will have a unique pts.
Patch Set #3: Use integer math to generate source pts. Added a
framerate parameter. Increased the default timebase to milliseconds to
remove the *2 everywhere.
John Koleszar [Wed, 20 Oct 2010 14:49:12 +0000 (10:49 -0400)]
ivfdec: support y4m output from raw input
The width and height needed to write the Y4M header can be found by
probing the stream with vpx_codec_peek_stream_info(). This also
has the consequence of supporting multiple codecs from raw files
with automatic detections, should we add additional codecs in the
future.
John Koleszar [Tue, 19 Oct 2010 21:20:17 +0000 (17:20 -0400)]
ivfdec: webm reader support
This patch enables ivfdec to decode WebM files. WebM demuxing is
provided by the Matthew Gregan's Nestegg library.
This patch also makes minor changes to the timebase->framerate
handling when doing Y4M output. For WebM files, the framerate is
guessed by looking at the first second of video. For IVF files,
the timebase=1/(2*fps) hack is still in place, but is only used
if the timebase denominator is less than 1000. This is in anticipation
of change I8d25b5b, which introduces the distinction between
framerate and timebase to ivfenc. In the case of high resolution
timebases, like 100ns, we would have to guess the framerate
like we do for WebM, but since WebM support in ivfenc will
deprecate IVF output, we just assume 30fps rather than writing the
lookahead code.
Solaris 10 requires -lposix4 to build successfully on gcc. I only have a
Sparc machine to test with on Solaris 10, but this change leaves
OpenSolaris x86 in a usable state w/ gnu-generic.
I am of the belief that this change should fix Solaris 10 on Sparc, but
will leave other Solaris architectures as is. If someone has an x86
Solaris 10 machine to test on, they may add x86-solaris-gcc to
libvpx/configure and give it a go.
John Koleszar [Mon, 25 Oct 2010 14:28:45 +0000 (10:28 -0400)]
NASM: trailing slash for ASFLAGS includes
Fix out-of-tree builds using NASM. NASM expects its include paths to
have a trailing slash. These aren't used used when doing in-tree builds
(./configure)
Johann [Mon, 18 Oct 2010 18:57:40 +0000 (14:57 -0400)]
reuse common loopfilter code
there were four versions for the regular and
macroblock loopfilters:
horizontal [y|uv]
vertical [y|uv]
this moves all the common code into 2 functions:
vp8_loop_filter_neon
vp8_mbloop_filter_neon
this provides no gain in performance. there's a bit
of jitter, but it trends down ~0.25-0.5%. however,
this is a huge gain maintenance. also, there is the
potential to drop some stack usage in the macroblock
loopfilter.
The primary goal is to allow a binary to be built which supports
NEON, but can fall back to non-NEON routines, since some Android
devices do not have NEON, even if they are otherwise ARMv7 (e.g.,
Tegra).
The configure-generated flags HAVE_ARMV7, etc., are used to decide
which versions of each function to build, and when
CONFIG_RUNTIME_CPU_DETECT is enabled, the correct version is chosen
at run time.
In order for this to work, the CFLAGS must be set to something
appropriate (e.g., without -mfpu=neon for ARMv7, and with
appropriate -march and -mcpu for even earlier configurations), or
the native C code will not be able to run.
The ASFLAGS must remain set for the most advanced instruction set
required at build time, since the ARM assembler will refuse to emit
them otherwise.
I have not attempted to make any changes to configure to do this
automatically.
Doing so will probably require the addition of new configure options.
Many of the hooks for RTCD on ARM were already there, but a lot of
the code had bit-rotted, and a good deal of the ARM-specific code
is not integrated into the RTCD structs at all.
I did not try to resolve the latter, merely to add the minimal amount
of protection around them to allow RTCD to work.
Those functions that were called based on an ifdef at the calling
site were expanded to check the RTCD flags at that site, but they
should be added to an RTCD struct somewhere in the future.
The functions invoked with global function pointers still are, but
these should be moved into an RTCD struct for thread safety (I
believe every platform currently supported has atomic pointer
stores, but this is not guaranteed).
The encoder's boolhuff functions did not even have _c and armv7
suffixes, and the correct version was resolved at link time.
The token packing functions did have appropriate suffixes, but the
version was selected with a define, with no associated RTCD struct.
However, for both of these, the only armv7 instruction they actually
used was rbit, and this was completely superfluous, so I reworked
them to avoid it.
The only non-ARMv4 instruction remaining in them is clz, which is
ARMv5 (not even ARMv5TE is required).
Considering that there are no ARM-specific configs which are not at
least ARMv5TE, I did not try to detect these at runtime, and simply
enable them for ARMv5 and above.
Finally, the NEON register saving code was completely non-reentrant,
since it saved the registers to a global, static variable.
I moved the storage for this onto the stack.
A single binary built with this code was tested on an ARM11 (ARMv6)
and a Cortex A8 (ARMv7 w/NEON), for both the encoder and decoder,
and produced identical output, while using the correct accelerated
functions on each.
I did not test on any earlier processors.
The code was not checking for frame sizes smaller than 3 bytes, and the
partition size checks might have failed if the input buffer was within
16MB of the top of the heap.
In addition, the reference count on the current frame buffer was not
being decremented on error, so after a small number of errors, no new
frame buffer could be found and it would run off the list of them.
Most of the code that actually uses these matrices indexes them as
if they were a single contiguous array, and coverity produces
reports about the resulting accesses that overflow the static
bounds of the first row.
This is perfectly legal in C, but converting them to actual [16]
arrays should eliminate the report, and removes a good deal of
extraneous indexing and address operators from the code.
John Koleszar [Thu, 14 Oct 2010 20:40:12 +0000 (16:40 -0400)]
Move firstpass motion map to stats packet
The first implementation of the firstpass motion map for motion
compensated temporal filtering created a file, fpmotionmap.stt,
in the current working directory. This was not safe for multiple
encoder instances. This patch merges this data into the first pass
stats packet interface, so that it is handled like the other
(numerical) firstpass stats.
The new stats packet is defined as follows:
Numerical Stats (16 doubles) -- 128 bytes
Motion Map -- 1 byte / Macroblock
Padding -- to align packet to 8 bytes
The fpmotionmap.stt file can still be generated for debugging
purposes in the same way that the textual version of the stats
are available (defining OUTPUT_FPF in firstpass.c)
Yunqing Wang [Thu, 21 Oct 2010 14:26:50 +0000 (10:26 -0400)]
Rewrite vp8_short_walsh4x4_sse2()
This rewriting reflects changes made in commit "Improve the
accuracy of forward walsh-hadamard transform". Since this function
is not called much, only a small encoder performance gain (~0.5% )
is seen.
Yunqing Wang [Mon, 18 Oct 2010 18:15:15 +0000 (14:15 -0400)]
Add SSE2 subtract functions
Instead of doing 8-bit data unpack and 16-bit subtraction, use
psubb to do 16 8-bit subtractions and pcmpgtb to preserve the
sign information. This does not bring noticable gain since
these functions are not called frequently.
Johann [Mon, 18 Oct 2010 17:23:39 +0000 (13:23 -0400)]
copy compiler warning fixes
generic version got fixed, but not the arm version. fixes:
vp8/encoder/arm/mcomp_arm.c: In function 'vp8_full_search_sadx3':
vp8/encoder/arm/mcomp_arm.c:1208: warning: pointer targets in passing
argument 5 of 'fn_ptr->sdx3f' differ in signedness
vp8/encoder/arm/mcomp_arm.c:1208: note: expected 'unsigned int *' but
argument is of type 'int *'
and another unsigned change to keep the files similar
Yaowu Xu [Fri, 15 Oct 2010 01:58:34 +0000 (18:58 -0700)]
change to make use of more trellis quantization
when a subsequent frame is encoded as an alt reference frame, it is
unlikely that any mb in current frame will be used as reference for
future frames, so we can enable quantization optimization even when
the RD constant is slightly rate-biased. The change has an overall
benefit between 0.1% to 0.2% bit savings on the test sets based on
vpxssim scores.
Yunqing Wang [Thu, 14 Oct 2010 18:25:03 +0000 (14:25 -0400)]
Fix one gcc compiler warning
../libvpx/vp8/encoder/bitstream.c: In function ‘pack_inter_mode_mvs’:
../libvpx/vp8/encoder/bitstream.c:1026: warning: array subscript has type ‘char’
Yunqing Wang [Thu, 14 Oct 2010 15:06:37 +0000 (11:06 -0400)]
Improve bounds checking in vp8_diamond_search_sadx4()
In order to know if all 4/8 neighbor points are within the bounds,
4 bounds checking are enough instead of checking 4 bounds for
each points (16/32 checkings). This improvement reduces cost of
vp8_diamond_search_sadx4() by 30%, and gives encoder a 1.5%
performance gain (test options: 1 pass, good, speed=4).
John Koleszar [Thu, 7 Oct 2010 05:39:16 +0000 (22:39 -0700)]
Centralize mb skip state calculation
This patch moves the scattered updates to the mb skip state
(mode_info_context->mbmi.mb_skip_coeff) to vp8_tokenize_mb. Recent
changes to the quantizer exposed a bug where if a macroblock
could be coded as a skip but isn't, the encoder would run the
loopfilter but the decoder wouldn't, causing a reference buffer
mismatch.
The loopfilter is controlled by a flag called dc_diff. The decoder
looks at the number of decoded coefficients when setting this flag.
The encoder sets this flag based on the skip state, since any
skippable macroblock should be transmitted as a skip. The coefficient
optimization pass (vp8_optimize_b()) could change the coefficients
such that a block that was not a skip becomes one. The encoder was
not updating the skip state in this situation for intra coded blocks.
The underlying issue predates it, but this bug was recently triggered
by enabling trellis quantization on the Y2 block in commit dcd29e3,
and by changing the quantizer range control in commit 305be4e.
There is currently no inexact version of this function, so do not
even compile it without EXACT_QUANT.
This will prevent someone from inadvertently trying to use it without
the proper EXACT_QUANT setup.
Remove INTRARDOPT #define and intra_rd_opt option.
This is just eliminating some cruft.
Although a number of variables are declared only when INTRARDOPT
is defined, they are used elsewhere without that protection, and
no longer just for intra RDO.
The intra_rd_opt flag was hard-coded to 1 and never checked.
Yaowu Xu [Wed, 6 Oct 2010 20:28:36 +0000 (13:28 -0700)]
optimize fast_quantizer c version
As the zbin and rounding constants are normalized, rounding effectively
does the zbinning, therefore the zbin operation can be removed. In
addition, the memset on the two arrays are no longer necessary.
Jan Kratochvil [Tue, 5 Oct 2010 17:15:08 +0000 (19:15 +0200)]
nasm: add configure support
yasm has to be preferred as currently nasm produces marginally less
efficient code (longer opcodes). Filed for nasm as:
https://sourceforge.net/tracker/?func=detail&atid=106208&aid=3037462&group_id=6208
OTOH package should be built always the same, no matter which additional
packages are / are not present on the system. As the package should be
built with nasm (as yasm may not be available) we should not use yasm
even if it is possibly available.
nasm >= approx. 2.09 is required for the nasm compilation as the former
versions had a section alignment bug.
Provide nasm compatibility. No binary change by this patch with yasm on
{x86_64,i686}-fedora13-linux-gnu. Few longer opcodes with nasm on
{x86_64,i686}-fedora13-linux-gnu have been checked as safe.
Jan Kratochvil [Sat, 31 Jul 2010 15:12:32 +0000 (17:12 +0200)]
nasm: avoid relative include paths
nasm does not automatically assume the source's directory also for its
include files.
Provide nasm compatibility. No binary change by this patch with yasm on
{x86_64,i686}-fedora13-linux-gnu. Few longer opcodes with nasm on
{x86_64,i686}-fedora13-linux-gnu have been checked as safe.
Jan Kratochvil [Mon, 4 Oct 2010 21:18:58 +0000 (23:18 +0200)]
nasm: address labels 'rel label' vice 'wrt rip'
nasm does not support `label wrt rip', it requires `rel label'. It is
still fully compatible with yasm.
Provide nasm compatibility. No binary change by this patch with yasm on
{x86_64,i686}-fedora13-linux-gnu. Few longer opcodes with nasm on
{x86_64,i686}-fedora13-linux-gnu have been checked as safe.
Jan Kratochvil [Mon, 4 Oct 2010 21:19:33 +0000 (23:19 +0200)]
nasm: match instruction length (movd/movq) to parameters
nasm requires the instruction length (movd/movq) to match to its
parameters. I find it more clear to really use 64bit instructions when
we use 64bit registers in the assembly.
Provide nasm compatibility. No binary change by this patch with yasm on
{x86_64,i686}-fedora13-linux-gnu. Few longer opcodes with nasm on
{x86_64,i686}-fedora13-linux-gnu have been checked as safe.
Yaowu Xu [Mon, 4 Oct 2010 17:58:42 +0000 (10:58 -0700)]
fixed a typo that mis-used Y plane stride for UV blocks.
Raised by Lei Yang, the Y plane stride was used for UV blocks.
This is clearly a typo. But as the comments in the code suggested
that this port of code has not been used yet, so the typo should
not have created any damage yet.
Yaowu Xu [Fri, 1 Oct 2010 03:41:37 +0000 (20:41 -0700)]
enable trellis quantization for 2nd order blocks
Experimented with different value for Y2_RD_MULT ranging f[1, 32],
without adapting the value to MB coding mode/frame type/Q value,
4 works out best among all values, providing overall 0.1% coding
gain on the test set.
Adrian Grange [Thu, 30 Sep 2010 09:06:09 +0000 (10:06 +0100)]
Changed defaults & range checking for AltRef params
Modified the range checking of parameters used in the
AltRef temporal filter (arnr-max-frames, arnr-strength,
arnr-type) and default values for each of them.
John Koleszar [Wed, 29 Sep 2010 17:04:04 +0000 (13:04 -0400)]
Fix loopfilter delta zero transitions
Loopfilter deltas are initialized to zero on keyframes in the decoder.
The values then persist from the previous frame unless an update bit
is set in the bitstream. This data is not included in the entropy
data saved by the 'refresh entropy' bit in the bitstream, so it is
effectively an additional contextual element beyond the 3 ref-frames
and the entropy data.
The encoder was treating this delta update bit as update-if-nonzero,
meaning that the value would be refreshed even if it hadn't changed,
and more significantly, if the correct value for the delta changed
to zero, the update wouldn't be sent, and the decoder would preserve
the last (presumably non-zero) value.
This patch updates the encoder to send an update only if the value
has changed from the previously transmitted value. It also forces the
value to be transmitted in error resilient mode, to account for lost
context in the event of lost frames.
Paul Wilkins [Wed, 29 Sep 2010 11:03:19 +0000 (12:03 +0100)]
Control of active min quantizer for two pass.
Create look up tables for controlling the active quantizer range.
Some initial tuning to improve quality circa 0.5% on test set.
Clean up of some stats output code
Fritz Koenig [Tue, 28 Sep 2010 19:01:34 +0000 (12:01 -0700)]
Optimizations on the loopfilters.
- Scheduling for Atom processors
- Combining of macros to allow for better interleaving
- Change from multiplies to adds for main filter
- Use of movhps/movlps to fill xmm registers without
shifting and orring
Johann [Tue, 28 Sep 2010 13:31:11 +0000 (09:31 -0400)]
update gitignore
this was excluding all .asm files when it should have just been .asm
files in the top level directory and .asm.s files lower down. also be
more restrictive on some other items, and run the whole thing through
sort to keep it organized