Deb Mukherjee [Fri, 27 Jul 2012 18:29:46 +0000 (11:29 -0700)]
Merges several experiments
The following five experiments are merged:
newentropy
newupdate
adaptive_entropy (also includes a couple of parameter changes
that improves results a little
in common/entropymode.c and encoder/modecosts.c
that were not merged from the internal branch)
newintramodes
expanded_coef_context
Jingning Han [Mon, 25 Jun 2012 19:26:09 +0000 (12:26 -0700)]
Adds hybrid transform
Adds ADST/DCT hybrid transform coding for Intra4x4 mode.
The ADST is applied to directions in which the boundary
pixels are used for prediction, while DCT applied to
directions without corresponding boundary prediction.
Adds enum TX_TYPE in b_mode_infor to indicate the transform
type used.
Make coding style consistent with google style.
Fixed the commented issues.
Experimental results in terms of bit-rate reduction:
derf: 0.731%
yt: 0.982%
std-hd: 0.459%
hd: 0.725%
the integer version has very good precision, the float version is no
longer useful. this commit also removes the experiment option from
configure script.
this commit removes a number of experiment options from configure
script. the associated experiments are already fully merged, the
options in configure script have no effect at all.
Ronald S. Bultje [Tue, 17 Jul 2012 00:49:37 +0000 (17:49 -0700)]
Fix bug in reference frame counting.
vp8_encode_inter_macroblock() is called in both pick_mb_modes() as
well as encode_sb(), thus the number of macroblocks in the counter
were twice as big as actual numbers. This doesn't affect output.
Yaowu Xu [Fri, 29 Jun 2012 00:03:29 +0000 (17:03 -0700)]
change to enable encoder to clamp UV motion vector
This is to avoid a rare encoder/decoder mismatch for MB using SPLITMV
mode. In decoder, the UV mv can be determined to need clamp, but the
flag is never set in encoder motion vector selection process, and the
clamp is not done in encoding in encoder.
Hui Su [Thu, 14 Jun 2012 02:03:31 +0000 (19:03 -0700)]
Add lossless compression mode.
This commit adds lossless compression capability to the experimental
branch. The lossless experiment can be enabled using --enable-lossless
in configure. When the experiment is enabled, the encoder will use
lossless compression mode by command line option --lossless, and the
decoder automatically recognizes a losslessly encoded clip and decodes
accordingly.
To achieve the lossless coding, this commit has changed the following:
1. To encode at lossless mode, encoder forces the use of unit
quantizer, i.e, Q 0, where effective quantization is 1. Encoder also
disables the usage of 8x8 transform and allows only 4x4 transform;
2. At Q 0, the first order 4x4 DCT/IDCT have been switched over
to a pair of forward and inverse Walsh-Hadamard Transform
(http://goo.gl/EIsfy), with proper scaling applied to match the range
of the original 4x4 DCT/IDCT pair;
3. At Q 0, the second order remains to use the previous
walsh-hadamard transform pair. However, to maintain the reversibility
in second order transform at Q 0, scaling down is applied to first
order DC coefficients prior to forward transform, and scaling up is
applied to the second order output prior to quantization. Symmetric
upscaling and downscaling are added around inverse second order
transform;
4. At lossless mode, encoder also disables a number of minor
features to ensure no loss is introduced, these features includes:
a. Trellis quantization optimization
b. Loop filtering
c. Aggressive zero-binning, rounding and zero-bin boosting
d. Mode based zero-bin boosting
Lossless coding test was performed on all clips within the derf set,
to verify that the commit has achieved lossless compression for all
clips. The average compression ratio is around 2.57 to 1.
(http://goo.gl/dEShs)
Adrian Grange [Mon, 25 Jun 2012 23:23:58 +0000 (16:23 -0700)]
Added Prediction Filter to Mode Selection
Added the ability to optionally filter the prediction data
when inter modes are selected (excludes SPLITMV, for now).
The mode selection loop considers both the filtered and
non-filtered prediction data when choosing mode. The filter
can be turned on/off at the frame-level, or signaled for
each MB.
Deb Mukherjee [Tue, 5 Jun 2012 22:25:07 +0000 (15:25 -0700)]
Adaptive entropy coding of coefficients, modes, mv.
This patch incorporates adaptive entropy coding of coefficient tokens,
and mode/mv information based on distributions encountered in a frame.
Specifically, there is an initial forward update to the probabilities
in the bitstream as before for coding the symbols in the frame, however
at the end of decoding each frame, the forward update to the
probabilities is reverted and instead the probabilities are updated
towards the actual distributions encountered within the frame.
The amount of update is weighted by the number of hits within each
context.
Results on derf/hd/std-hd are all up by 1.6%.
On derf, the most of the gains come from coefficients, however for the
hd and std-hd sets, the most of the gains come from the mode/mv
information updates.
Yaowu Xu [Thu, 24 May 2012 19:02:12 +0000 (12:02 -0700)]
changed to use a fixed number for update probabity
This commit is to remove two arrays, which contain the probabilities
of how likely each probability in coef_probs table is updated. The
commit changed to use a fixed number "252".
Surprisedly, the overall impact on quality is close to zero, which
basically says the two big static arrays are not helpful at all.
derf: -0.016%, -0.020%
std-hd: 0.000%, -0.013%
yt: -0.022%, +0.007%
yt-hd: -0.038%, +0.034%
Yaowu Xu [Tue, 8 May 2012 19:38:39 +0000 (12:38 -0700)]
Added the ability to accumulate coef stats across encodings
This commit added the ability to accumulate the coef stats across
different encodings using an intermediate binary stats files. The
accumulation happens only the binary stats file exists in current
directory. The encoder needs to be built with "ENTROPY_STATS" to
allow the output. The commit also fixed a few formating issues in
output stats file.
Paul Wilkins [Wed, 30 May 2012 11:54:34 +0000 (12:54 +0100)]
Merge of further two pass rc clean up and adjustments.
Changes to calculation of sr_coded_error to include 0,0 case.
Experimental use of sr_coded_error in calculating correction factor
for estimating the allowable Q range.
Reinstated some code needed for calculating section_intra_rating.
Add flash detection in calculation of KF boost
Increased tolerance in testing candidate key frames (needed with
longer motion search as this tends to slightly increase inter %.
Zbin changes for 8x8.
Other minor adjustments, refactoring and bug fixes.
Reinstated some motion break out clauses in boost loop
as their removal hurt a few 50fps clips badly in the std set.
It may be possible to remove them again later if a better way
can be found of preventing overly long gf intervals.
Paul Wilkins [Wed, 30 May 2012 10:29:49 +0000 (11:29 +0100)]
Rate control fix.
This fix addresses some problems with very complex clips like
handling of flashes on clips like crew (which was made worse
by an earlier patch (derf and std-hd)).
Yaowu Xu [Wed, 16 May 2012 19:07:46 +0000 (12:07 -0700)]
changed the way that default probs for 8x8 is set.
The commit changed how baseline 8x8 coefficient probabilities are
initialized, to be consistent with the initialization of baseline
4x4 coefficient probabilities.
The commit does not have any effect on compression.
Ronald S. Bultje [Tue, 15 May 2012 00:39:42 +0000 (17:39 -0700)]
Rewrite reference frame costing in the RD loop.
I now see I didn't write a very long description, so let's do it
here then. We took a pretty big quality hit (0.1-0.2%) from my
recent fix of the inversion of arguments to vp8_cost_bit() in the
RD reference frame costing. I looked into it and basically the
costing prevented us from switching reference frames. This is of
course silly, since each frame codes its own prob_intra_coded, so
using last frame cost indications as a limiting factor can never
be right.
Here, I've rewritten that code to estimate costings based partially
on statistics from progress on current frame encoding. Overall,
this gives us a ~0.2%-0.3% improvement over what we had previously
before my argument-inversion-fix, and thus about ~0.4% over current
git (on derf-set), and a little more (0.5-1.0%) on HD/STD-HD/YT.
Yaowu Xu [Wed, 9 May 2012 16:31:14 +0000 (09:31 -0700)]
Changed to use integer 8x8 dct
The commit added an integer version of 8x8 forward DCT, based on the
orginal forward DCT from VP6. The constants, roundings, and shifts
were adjusted to improve the accuracy. The latest patch has a very
similar accuracy in term of round trip error against the floating
point version.
It should be noted here that the purpose of the patch is to help
encoding speed and facilitate all other experiments. There will be
futher review in combination with inverse DCT before finalization.
configure with "--enable--int_8x8fdct" to use the integer version
Paul Wilkins [Tue, 15 May 2012 13:58:13 +0000 (14:58 +0100)]
Firstpass.c refactoring
Removed unused function.
Added tentative code to take error score of an older frame
into account when calculating Q range. However, for now
it is disabled pending merging other changes and testing.
Paul Wilkins [Mon, 14 May 2012 14:13:26 +0000 (15:13 +0100)]
Two pass refactoring continued.
Remove testing of whether we estimate that it will be possible
to code an arf at a lower Q than the ambient Q. This adds quite
a bit of extra code and complexity for marginal gain.
Factored out some code relating to ARNR selection to a separate
function as this is likely to be changed / simplified soon.
Paul Wilkins [Mon, 14 May 2012 13:54:54 +0000 (14:54 +0100)]
Two pass rc refactoring.
Removed odd *100 >> 4 factor from boost calculations. Not all the
calculations exactly match what was there before so there may be
some minor impact on results.
Some other minor tidying up in regard to coding conventions.
The specific values of factors and thresholds will likely change as
part of subsequent patches.
Ronald S. Bultje [Sat, 12 May 2012 00:48:20 +0000 (17:48 -0700)]
Don't use compound prediction for golden frames based on alt-ref frames.
These frames can force reference frame (arf), mode (zeromv) and skip,
which means that if we use compound prediction (i.e. arf+last), we
might use a blend of a perfect (arf) and an imperfect (last) predictor,
leading to semi-garbage display and thus a huge drop in SSIM/PSNR (up
to 10dB for some frames I analyzed).
Paul Wilkins [Fri, 11 May 2012 17:07:33 +0000 (18:07 +0100)]
First pass overhaul preparatory change.
This is the first patch in a series of changes to the first
pass code. (Broken down for ease of testing/merging/review).
This patch introduces a new stats element "sr_coded_error".
This is the coded error recorded vs the second reference
frame (which is updated such that it lags by at least one frame).
No use is made of the new structure in this change so this patch
should have no material effect.
Removed some ifdefs and deprecated code (#if NEW_BOOST).
Removed twopass.gf_decay_rate (not used any more)
Yaowu Xu [Thu, 10 May 2012 00:19:16 +0000 (17:19 -0700)]
Reversible WHT pair
This commit changed the forward and the inverse 4x4 Walsh Hadamard
transform to a new pair, where the inverse transform can pefectly
reconstuct the input to forward transform. It also does so without
changing the input and output value range. Even more, it does not
change the complexity of the transforms.
While it was not expected to improve the results of our current test,
it does improve std-hd set by 0.2% on all metrics. No change on derf.
Yaowu Xu [Wed, 2 May 2012 20:53:15 +0000 (13:53 -0700)]
a number of fixes to entropy stats collection
1. block types
There are only three types of blocks for 8x8 transformed MBs, i.e. Y
block with DC does not exist for 8x8 transformed MBs as all MB using
8x8 transform have 2nd order haar transform. This commit introduced
a new macro BLOCK_TYPES_8X8 to reflect such fact.
2. context counters
This commit also fixed the mixed of context_counters between 4x4 and
8x8 transformed MBs. The mixed use of the counters leads me to think
the existing the context probabilities were not properly generated
from 8x8 transformed MBs.
3. redundant collecting in recoding
The commit also corrected the code that accumulates entropy stats by
making sure stats only collected for final packing, not during the
recode loop
Deb Mukherjee [Wed, 21 Mar 2012 22:22:21 +0000 (15:22 -0700)]
Expanding the coefficient encoding contexts
This patch expands the set of prev contexts used for video coding
from 3 to 4.
There is a small improvement of the order of 0.08% for derf and
0.15% on the HD set. The tests were rerun after the various merges
last week. There are two columns in each test - the first are the
results with the mbskip change, and the second with expanded contexts
added on top of that.
Yaowu Xu [Wed, 2 May 2012 02:54:19 +0000 (19:54 -0700)]
slight adjustment to coef band definition
This commit adjusted slightly the 4x4 coefficents band definition to
better classify coefficients with similar distributions and usages.
It helps derf set about .1%, it is alos slightly positive for std-hd
set, where 4x4 blocks are used less frequently.
projects / libvpx / log