Paul Wilkins [Mon, 15 Nov 2010 17:47:12 +0000 (17:47 +0000)]
Bad cost tables used in ARNR filtering.
The use of incorrect mv costing tables in the ARNR sub-pel
filtering code led to corruption of the altref buffer in some cases,
particularly at low data rates.
The average gain from this fix is about 0.3% but there are a few
extreme cases where nasty and visible artifacts manifested and
for these few data points the improvement is > 10%.
Paul Wilkins [Mon, 8 Nov 2010 15:28:54 +0000 (15:28 +0000)]
Tuning for the more exact quantizer.
Small changes to the default zero bin and rounding tables.
Though the tables are currently the same for the Y1 and Y2 cases
I have left them as separate tables in case we want to tune this later.
There is now some adjustment of the zbin based on the prediction mode.
Previously this was restricted to an adjustment for gf/arf 0,0 MV.
The exact quantizer now marginal outperforms and is the default.
John Koleszar [Thu, 4 Nov 2010 20:59:26 +0000 (16:59 -0400)]
fix integer promotion bug in partition size check
The check '(user_data_end - partition < partition_size)' must be
evaluated as a signed comparison, but because partition_size was
unsigned, the LHS was promoted to unsigned, causing an incorrect
result on 32-bit. Instead, check the upper and lower bounds of
the segment separately.
John Koleszar [Thu, 4 Nov 2010 19:05:45 +0000 (15:05 -0400)]
improve average framerate calculation
Change Ice204e86 identified a problem with bitrate undershoot due to
low precision in the timestamps passed to the library. This patch
takes a different approach by calculating the duration of this frame
and passing it to the library, rather than using a fixed duration
and letting the library average it out with higher precision
timestamps. This part of the fix only applies to vpxenc.
This patch also attempts to fix the problem for generic applications
that may have made the same mistake vpxenc did. Instead of
calculating this frame's duration by the difference of this frame's
and the last frame's start time, we use the end times instead. This
allows the framerate calculation to scavenge "unclaimed" time from
the last frame. For instance:
Yaowu Xu [Wed, 3 Nov 2010 19:56:31 +0000 (12:56 -0700)]
Increase the resolution of default timebase
The old value 1000 was too low, which caused the effective duration and
frame rate calculation to have an 1% error for typical 30 frame/second
inputs. Symptom of the issue has been that most 2 pass encodings were
undershooting target bit rate by 1% or so for 30 fps input.
John Koleszar [Wed, 3 Nov 2010 17:58:40 +0000 (13:58 -0400)]
vpxenc: require width and height for raw streams
Defaulting to 320x240 for raw streams is arbitrary and error-prone.
Instead, require that the width and height be set manually if they
can't be parsed from the input file.
John Koleszar [Tue, 2 Nov 2010 13:11:57 +0000 (09:11 -0400)]
fix pipe support on windows
STDIO streams are opened in text mode by default on Windows. This patch
changes the stdin/stdout streams to be in binary mode if they are being
used for I/O from the vpxenc or vpxdec tools.
Fixes issue #216. Thanks to mw AT hesotech.de for the fix.
This eliminates a large set of warnings exposed by the Mozilla build
system (Use of C++ comments in ISO C90 source, commas at the end of
enum lists, a couple incomplete initializers, and signed/unsigned
comparisons).
It also eliminates many (but not all) of the warnings expose by newer
GCC versions and _FORTIFY_SOURCE (e.g., calling fread and fwrite
without checking the return values).
There are a few spurious warnings left on my system:
../vp8/encoder/encodemb.c:274:9: warning: 'sz' may be used
uninitialized in this function
gcc seems to be unable to figure out that the value shortcut doesn't
change between the two if blocks that test it here.
../vp8/encoder/onyx_if.c:5314:5: warning: comparison of unsigned
expression >= 0 is always true
../vp8/encoder/onyx_if.c:5319:5: warning: comparison of unsigned
expression >= 0 is always true
This is true, so far as it goes, but it's comparing against an enum,
and the C standard does not mandate that enums be unsigned, so the
checks can't be removed.
Fritz Koenig [Wed, 27 Oct 2010 19:50:16 +0000 (12:50 -0700)]
postproc: Tweaks to line drawing and blending.
Turned down the blending level to make colored blocks obscure
the video less.
Not blending the entire block to give distinction to macro
block edges.
Added configuration so that macro block blending function can
be optimized.
Change to constrain line as to when dx and dy are computed.
Now draw two lines to form an arrow.
Frank Galligan [Wed, 27 Oct 2010 15:28:56 +0000 (11:28 -0400)]
Output the PSNR for the entire file.
If --psnr option is enabled vpxenc will output PSNR values for the
entire file. Added a \n before final output to make sure the output
is on its own line. Overall and Avg psnr matches the values written
to opsnr.stt file.
This eliminates a large set of warnings exposed by the Mozilla build
system (Use of C++ comments in ISO C90 source, commas at the end of
enum lists, a couple incomplete initializers, and signed/unsigned
comparisons).
It also eliminates many (but not all) of the warnings expose by newer
GCC versions and _FORTIFY_SOURCE (e.g., calling fread and fwrite
without checking the return values).
There are a few spurious warnings left on my system:
../vp8/encoder/encodemb.c:274:9: warning: 'sz' may be used
uninitialized in this function
gcc seems to be unable to figure out that the value shortcut doesn't
change between the two if blocks that test it here.
../vp8/encoder/onyx_if.c:5314:5: warning: comparison of unsigned
expression >= 0 is always true
../vp8/encoder/onyx_if.c:5319:5: warning: comparison of unsigned
expression >= 0 is always true
This is true, so far as it goes, but it's comparing against an enum, and the C
standard does not mandate that enums be unsigned, so the checks can't be
removed.
Fritz Koenig [Wed, 27 Oct 2010 19:50:16 +0000 (12:50 -0700)]
postproc: Tweaks to line drawing and blending.
Turned down the blending level to make colored blocks obscure
the video less.
Not blending the entire block to give distinction to macro
block edges.
Added configuration so that macro block blending function can
be optimized.
Change to constrain line as to when dx and dy are computed.
Now draw two lines to form an arrow.
Frank Galligan [Wed, 27 Oct 2010 15:28:56 +0000 (11:28 -0400)]
Output the PSNR for the entire file.
If --psnr option is enabled vpxenc will output PSNR values for the
entire file. Added a \n before final output to make sure the output
is on its own line. Overall and Avg psnr matches the values written
to opsnr.stt file.
Yunqing Wang [Wed, 27 Oct 2010 12:45:24 +0000 (08:45 -0400)]
Full search SAD function optimization in SSE4.1
Use mpsadbw, and calculate 8 sad at once. Function list:
vp8_sad16x16x8_sse4
vp8_sad16x8x8_sse4
vp8_sad8x16x8_sse4
vp8_sad8x8x8_sse4
vp8_sad4x4x8_sse4
(test clip: tulip)
For best quality mode, this gave encoder a 5% performance boost.
For good quality mode with speed=1, this gave encoder a 3%
performance boost.
John Koleszar [Wed, 27 Oct 2010 14:05:55 +0000 (10:05 -0400)]
vpxenc: add unique track id
MKV requires a unique(ish) TrackID element in the track info header.
Instead of the current hard-coded ID, take a hash of the video track
and use that. This value is not written in the deterministic output
mode, despite being a deterministic value itself, to give flexibility
to change the hash algorithm and not affect bisecting across the
change.
Johann [Wed, 27 Oct 2010 15:21:02 +0000 (11:21 -0400)]
fix implicit declarations
ARM used to explicitly remove this file from the build. With the RTCD
changes, that's no longer possible. These errors also exist for x86 w/o
RTCD, but that's not the default configuration
John Koleszar [Tue, 26 Oct 2010 19:34:16 +0000 (15:34 -0400)]
Add half-pixel variance RTCD functions
NEON has optimized 16x16 half-pixel variance functions, but they
were not part of the RTCD framework. Add these functions to RTCD,
so that other platforms can make use of this optimization in the
future and special-case ARM code can be removed.
A number of functions were taking two variance functions as
parameters. These functions were changed to take a single
parameter, a pointer to a struct containing all the variance
functions for that block size. This provides additional flexibility
for calling additional variance functions (the half-pixel special
case, for example) and by initializing the table for all block sizes,
we don't have to construct this function pointer table for each
macroblock.
John Koleszar [Tue, 26 Oct 2010 20:22:22 +0000 (16:22 -0400)]
vpxenc: add deterministic output option
By baking the version number into the output file, a hash of the file
will vary from commit to commit, even if the output is otherwise bit
exact. Add a -D option to suppress this behavior, for use when
bisecting or other debugging.
John Koleszar [Tue, 26 Oct 2010 15:37:23 +0000 (11:37 -0400)]
make vp8_recon16x16mb{,y} RTCD functions
ARM NEON has a platform specific version of vp8_recon16x16mb, though
it's just a stub to extract the various parameters from the
MACROBLOCKD struct and pass them to vp8_recon16x16mb_neon(). Using
that function's prototype directly will be a better long term solution,
but it's quite an invasive change.
John Koleszar [Tue, 26 Oct 2010 14:46:31 +0000 (10:46 -0400)]
make arm hex search the generic implementation
The ARM version of vp8_hex_search() is a faster implementation
of the same algorithm. Since it doesn't use any ARM specific
code, it can be made the default implementation. This removes
a linking error.
John Koleszar [Tue, 26 Oct 2010 13:51:35 +0000 (09:51 -0400)]
arm: move unrolled loops back to generic code
Some of the ARM functions differed from their generic counterparts
only by unrolling their loops. Since this change may be useful
on other platforms, or might even supercede the looped version
in the generic case, move it back to the generic file.
This code is left under #if ARCH_ARM for now, but it may be worth
considering a different (possibly new) conditional for these. If
it turns out that this should be runtime selectable, these
functions will have to move to the RTCD infrastructure. Don't want
to take that step at this time without more profile data.
John Koleszar [Tue, 26 Oct 2010 13:37:44 +0000 (09:37 -0400)]
arm: remove duplicate functions
These functions were true duplicates of functions present in the
generic code. This fixes some of the link errors when building
with --enable-shared --enable-pic.
John Koleszar [Fri, 22 Oct 2010 03:40:42 +0000 (20:40 -0700)]
vpxdec: rework default output parameters
This patch reworks the default behavior of the tool to output Y4M
instead of writing individual raw frames. The relevant controls are
now:
--yv12, --i420 - These options change the output format to be
raw planar data. The output will be Y4M unless
one of these options is specified.
--flipuv - Swaps the chroma planes. Works with Y4M output.
-o, --output - Sets the output filename. Defaults to stdout if
not specified. Supports escape character
expansion for frame width (%w) height (%h) and
sequence number (%1..%9). The --prefix option
has been removed in favor of this escape
expansion.
Since the output defaults to stdout if -o is not specified, an
error will be thrown if stdout is not connected to a pipe. This
can be overridden by specifying '-o -'.
John Koleszar [Thu, 21 Oct 2010 20:53:52 +0000 (16:53 -0400)]
usage: fix horizontal alignment of options
When showing the command usage information for vpxenc and vpxdec,
options with both a short and long version that do not take an
argument were not properly aligned.
John Koleszar [Thu, 21 Oct 2010 19:02:10 +0000 (15:02 -0400)]
rename ivf{enc,dec} to vpx{enc,dec}
The new WebM output support should be preferred to IVF, but we can't
change the default behavior of the ivf* tools. There are a few other
default behaviors for these tools that are counterintuitive for
historical reasons, and changing the binary name provides the
opportunity to clean those up as well. This patch takes the first
step by renaming the binaries.
Frank Galligan [Wed, 6 Oct 2010 16:51:00 +0000 (12:51 -0400)]
Fixed the timebase parameter of ivfenc.
Ivfenc will use timebase if it is set. If it is not set ivfenc will
still double the timebase so altref frames will have a unique pts.
Patch Set #3: Use integer math to generate source pts. Added a
framerate parameter. Increased the default timebase to milliseconds to
remove the *2 everywhere.
John Koleszar [Wed, 20 Oct 2010 14:49:12 +0000 (10:49 -0400)]
ivfdec: support y4m output from raw input
The width and height needed to write the Y4M header can be found by
probing the stream with vpx_codec_peek_stream_info(). This also
has the consequence of supporting multiple codecs from raw files
with automatic detections, should we add additional codecs in the
future.
John Koleszar [Tue, 19 Oct 2010 21:20:17 +0000 (17:20 -0400)]
ivfdec: webm reader support
This patch enables ivfdec to decode WebM files. WebM demuxing is
provided by the Matthew Gregan's Nestegg library.
This patch also makes minor changes to the timebase->framerate
handling when doing Y4M output. For WebM files, the framerate is
guessed by looking at the first second of video. For IVF files,
the timebase=1/(2*fps) hack is still in place, but is only used
if the timebase denominator is less than 1000. This is in anticipation
of change I8d25b5b, which introduces the distinction between
framerate and timebase to ivfenc. In the case of high resolution
timebases, like 100ns, we would have to guess the framerate
like we do for WebM, but since WebM support in ivfenc will
deprecate IVF output, we just assume 30fps rather than writing the
lookahead code.
Solaris 10 requires -lposix4 to build successfully on gcc. I only have a
Sparc machine to test with on Solaris 10, but this change leaves
OpenSolaris x86 in a usable state w/ gnu-generic.
I am of the belief that this change should fix Solaris 10 on Sparc, but
will leave other Solaris architectures as is. If someone has an x86
Solaris 10 machine to test on, they may add x86-solaris-gcc to
libvpx/configure and give it a go.
John Koleszar [Mon, 25 Oct 2010 14:28:45 +0000 (10:28 -0400)]
NASM: trailing slash for ASFLAGS includes
Fix out-of-tree builds using NASM. NASM expects its include paths to
have a trailing slash. These aren't used used when doing in-tree builds
(./configure)
Johann [Mon, 18 Oct 2010 18:57:40 +0000 (14:57 -0400)]
reuse common loopfilter code
there were four versions for the regular and
macroblock loopfilters:
horizontal [y|uv]
vertical [y|uv]
this moves all the common code into 2 functions:
vp8_loop_filter_neon
vp8_mbloop_filter_neon
this provides no gain in performance. there's a bit
of jitter, but it trends down ~0.25-0.5%. however,
this is a huge gain maintenance. also, there is the
potential to drop some stack usage in the macroblock
loopfilter.
The primary goal is to allow a binary to be built which supports
NEON, but can fall back to non-NEON routines, since some Android
devices do not have NEON, even if they are otherwise ARMv7 (e.g.,
Tegra).
The configure-generated flags HAVE_ARMV7, etc., are used to decide
which versions of each function to build, and when
CONFIG_RUNTIME_CPU_DETECT is enabled, the correct version is chosen
at run time.
In order for this to work, the CFLAGS must be set to something
appropriate (e.g., without -mfpu=neon for ARMv7, and with
appropriate -march and -mcpu for even earlier configurations), or
the native C code will not be able to run.
The ASFLAGS must remain set for the most advanced instruction set
required at build time, since the ARM assembler will refuse to emit
them otherwise.
I have not attempted to make any changes to configure to do this
automatically.
Doing so will probably require the addition of new configure options.
Many of the hooks for RTCD on ARM were already there, but a lot of
the code had bit-rotted, and a good deal of the ARM-specific code
is not integrated into the RTCD structs at all.
I did not try to resolve the latter, merely to add the minimal amount
of protection around them to allow RTCD to work.
Those functions that were called based on an ifdef at the calling
site were expanded to check the RTCD flags at that site, but they
should be added to an RTCD struct somewhere in the future.
The functions invoked with global function pointers still are, but
these should be moved into an RTCD struct for thread safety (I
believe every platform currently supported has atomic pointer
stores, but this is not guaranteed).
The encoder's boolhuff functions did not even have _c and armv7
suffixes, and the correct version was resolved at link time.
The token packing functions did have appropriate suffixes, but the
version was selected with a define, with no associated RTCD struct.
However, for both of these, the only armv7 instruction they actually
used was rbit, and this was completely superfluous, so I reworked
them to avoid it.
The only non-ARMv4 instruction remaining in them is clz, which is
ARMv5 (not even ARMv5TE is required).
Considering that there are no ARM-specific configs which are not at
least ARMv5TE, I did not try to detect these at runtime, and simply
enable them for ARMv5 and above.
Finally, the NEON register saving code was completely non-reentrant,
since it saved the registers to a global, static variable.
I moved the storage for this onto the stack.
A single binary built with this code was tested on an ARM11 (ARMv6)
and a Cortex A8 (ARMv7 w/NEON), for both the encoder and decoder,
and produced identical output, while using the correct accelerated
functions on each.
I did not test on any earlier processors.
The code was not checking for frame sizes smaller than 3 bytes, and the
partition size checks might have failed if the input buffer was within
16MB of the top of the heap.
In addition, the reference count on the current frame buffer was not
being decremented on error, so after a small number of errors, no new
frame buffer could be found and it would run off the list of them.
Most of the code that actually uses these matrices indexes them as
if they were a single contiguous array, and coverity produces
reports about the resulting accesses that overflow the static
bounds of the first row.
This is perfectly legal in C, but converting them to actual [16]
arrays should eliminate the report, and removes a good deal of
extraneous indexing and address operators from the code.
John Koleszar [Thu, 14 Oct 2010 20:40:12 +0000 (16:40 -0400)]
Move firstpass motion map to stats packet
The first implementation of the firstpass motion map for motion
compensated temporal filtering created a file, fpmotionmap.stt,
in the current working directory. This was not safe for multiple
encoder instances. This patch merges this data into the first pass
stats packet interface, so that it is handled like the other
(numerical) firstpass stats.
The new stats packet is defined as follows:
Numerical Stats (16 doubles) -- 128 bytes
Motion Map -- 1 byte / Macroblock
Padding -- to align packet to 8 bytes
The fpmotionmap.stt file can still be generated for debugging
purposes in the same way that the textual version of the stats
are available (defining OUTPUT_FPF in firstpass.c)
projects / libvpx / log