vc2_conformance.encoder: Internal VC-2 encoder

The vc2_conformance.encoder module implements a simple VC-2 encoder which is used to produce test streams for conformance testing purposes (see vc2_conformance.test_cases). It is extremely slow and performs only simple picture compression, but is sufficiently flexible to support all VC-2 coding modes.

The encoder is principally concerned with carrying out the following tasks:

This module does not generate serialised VC-2 bitstreams in binary format. Instead, it generates a bitstream description data structure which may subsequently be serialised by the bitstream serialiser in the vc2_conformance.bitstream module. This design allows the generated bitstream to be more easily manipulated prior to serialisation if required for a particular test case.

Usage

The encoder behaviour is controlled by a CodecFeatures dictionary. This specifies picture/video format to be compressed (e.g. resolution etc.) along with the coding options (e.g. wavelet transform and bitrate). See the vc2_conformance.codec_features module for more details. There are essentially two modes of operation, depending on the value of CodecFeatures["lossless"]:

  • Lossless mode: variable bitrate, qindex is always 0.

  • Lossy mode: fixed bit rate, variable qindex.

A series of pictures may be encoded into a VC-2 sequence using the vc2_conformance.encoder.make_sequence() function, and then serialised into a binary bitstream as illustrated below:

>>> # Define the video format to be encoded and basic coding options
>>> from vc2_conformance.codec_features import CodecFeatures
>>> codec_features = CodecFeatures(...)

>>> # Encode a series of pictures
>>> from vc2_conformance.encoder import make_sequence
>>> pictures = [
...     {"Y": ..., "C1": ..., "C2": ..., "pic_num": 100},
...     # ...
... ]
>>> sequence = make_sequence(codec_features, pictures)

>>> # Serialise to a file
>>> from vc2_conformance.bitstream import (
...     Stream,
...     autofill_and_serialise_stream,
... )
>>> with open("bitstream.vc2", "wb") as f:
...     autofill_and_serialise_stream(f, Stream(sequences=[sequence]))

Bitstream conformance

This encoder will produce bitstreams conformant with the VC-2 specification whenever the coding parameters specified represent a legal combination.

In some cases, invalid coding options will cause the encoder to fail and raise an exception in vc2_conformance.encoder.exceptions. For example, if lossless coding and the low delay profile are requested simultaneously.

In other cases, the encoder will produce a bitstream, however this bitstream will be non-conformant. For example, if the clean area defined is larger than the frame size, the encoder will ignore this inconsistency of metadata and produce a (non-conformant) bitstream anyway.

When conformant bitstreams are required, it is the responsibility of the user of the encoder to ensure that the provided CodecFeatures are valid. In practice, the easiest way to do this is to check for exceptions when the encoder is used then use the bitstream validator (vc2_conformance.decoder) to validate the generated bitstream.

Exceptions

The exceptions defined in vc2_conformance.encoder.exceptions derive from UnsatisfiableCodecFeaturesError and are thrown when the presented encoder configuration makes encoding impossible. These exceptions provide detailed explanations of why encoding was not possible.

exception UnsatisfiableCodecFeaturesError

Base class for exceptions thrown by the encoder when it is unable to generate a stream in the desired format due to some invalid CodecFeatures configuration.

explain()

Produce a detailed human readable explanation of the failure.

Should return a string which can be re-linewrapped by vc2_conformance.string_utils.wrap_paragraphs().

The first line will be used as a summary when the exception is printed using str().

Sequence header generation

The vc2_conformance.encoder.sequence_header module contains routines for encoding a set of video format and codec parameters into sequence headers.

The make_sequence_header_data_unit() function is used to generate sequence headers by the encoder:

make_sequence_header_data_unit(codec_features)

Create a DataUnit object containing a sequence header which sensibly encodes the features specified in CodecFeatures dictionary provided.

Parameters
codec_featuresCodecFeatures
Returns
data_unitDataUnit
Raises
IncompatibleLevelAndVideoFormatError

In practice there are often many potential sequence header encodings for a given set of video parameters. For example, when a video format closely matches a predefined base video format, the various custom_*_flag overrides may largely be omitted. This is optional, however, and an encoder is free to use these overrides explicitly even when they’re not required.

The make_sequence_header_data_unit() function always attempts to use the most compact encoding it can. Some test cases, however may wish to use less compact encodings and so to support this the iter_sequence_headers() function is provided:

iter_sequence_headers(codec_features)

Generate a series of SequenceHeader objects which encode the video format specified in CodecFeatures dictionary provided.

This generator will start with an efficient encoding of the required features, built on the most closely matched base video format. This will be followed by successively less efficient encodings (i.e. using more custom fields) but the same (best-matched) base video format. After this, encodings based on other base video formats will be produced (again starting with the most efficient encoding for each format first).

This generator may output no items if the VC-2 level specified does not permit the format given.

Parameters
codec_featuresCodecFeatures
Yields
sequence_headerSequenceHeader

Picture encoding & compression

The vc2_conformance.encoder.pictures module contains simple routines for compressing pictures in a VC-2 bitstream.

The picture encoding behaviour used by the encoder is encapsulated by the make_picture_data_units() function which turns a series of pictures (given as raw pixel values) into a series of DataUnits:

make_picture_data_units(codec_features, picture, minimum_qindex=0, minimum_slice_size_scaler=1)

Create a seires of one or more DataUnits containing a compressed version of the supplied picture.

When codec_features["fragment_slice_count"] is 0, a single picture parse data unit will be produced. otherwise a series of two or more fragment parse data units will be produced.

A simple wrapper around make_picture_parse_data_unit() and make_fragment_parse_data_units().

Parameters
codec_featuresCodecFeatures
picture{“Y”: [[s, …], …], “C1”: …, “C2”: …, “pic_num”: int}

The picture to be encoded. This picture will be compressed using a simple VC-2 encoder implementation. It does not necessarily produce the most high-quality encodings. If pic_num is omitted, picture_number fields will be omitted in the output.

minimum_qindexint

Specifies the minimum quantization index to be used. Must be 0 for lossless codecs.

minimum_slice_size_scalerint

Specifies the minimum slice_size_scaler to be used for high quality pictures. Ignored in low delay mode.

Returns
data_units[vc2_conformance.bitstream.DataUnit, …]

Encoding algorithm

Depending on the lossy/lossless coding mode chosen, one of two simple algorithms is used.

Lossless mode

In lossless mode, every slice’s qindex will be set to 0 (no quantization) and all transform coefficients will be coded verbatim (though trailing zeros will be coded implicitly).

Slices will be sized as large as necessary, though as small as possible.

The smallest slice_size_scaler possible will be used for each coded picture independently.

Note

In principle, lossless modes may occasionally make use of quantization to achieve better compression. For example where all transform coefficients are a multiple of the same quantisation factor. This encoder, however, does not do this.

Lossy mode

In lossy mode the qindex for each slice is chosen on a slice-by-slice basis. The encoder tests quantization indices starting at zero and stopping when the transform coefficients fit into the slice.

Slices are sized such that the picture slice data in the bitstream totals CodecFeatures["picture_bytes"].

For the high quality profile, the smallest slice_size_scaler which can encode a slice where a single component consumes a whole slice is used for every picture.

Warning

The total size of picture slice data may differ from CodecFeatures["picture_bytes"] by up to slice_size_scaler bytes (for high quality profile formats) or one byte (for low delay profile formats). This will occur when the number of bytes (or multiple of slice_size_scaler bytes) is not exactly divisible by the required number of picture bytes.

Warning

The total number of bytes used to encode each picture, once other coding overheads (such as headers) will be higher than CodecFeatures["picture_bytes"].

Note

This codec may not always produce highest quality pictures possible in lossy modes. For example, sometimes chosing higher quantisation indices can produce fewer coding artefacts, particularly in concatenated coding applications. Similarly, higher picture quality may sometimes be obtained by setting later transform coefficients to zero enabling a lower quantization index to be used. Other more sophisticated schemes may also directly tweak transform coefficients.

Use of pseudocode

This module uses the pseudocode-derived vc2_conformance.pseudocode.picture_encoding module for its forward-DWT and vc2_conformance.pseudocode.quantization for quantization. Other pseudocode routines are also used where possible, for example for computing slice dimensions.

Sequence generation

The vc2_conformance.encoder.sequence module provides routines for constructing complete VC-2 sequences.

Principally, this module implements the make_sequence() function which produces vc2_conformance.bitstream.Sequence objects containing pictures compressed according to the required codec specifications. This is the main entry point to the encoder.

make_sequence(codec_features, pictures, *data_unit_patterns, **kwargs)

Generate a complete VC-2 bitstream based on the provided set of codec features and containing compressed versions of the specified set of pictures.

This function also takes a small number of additional parameters which override certain encoder behaviours as may be required by some test case generators.

Parameters
codec_featuresCodecFeatures
pictures[{“Y”: [[s, …], …], “C1”: …, “C2”: …, “pic_num”: int}, …]

The pictures to be encoded in the bitstream. If pic_num is omitted, picture_number fields will also be omitted in the output (and left for, e.g. vc2_conformance.bitstream.autofill_and_serialise_stream() to assign). See vc2_conformance.encoder.pictures for details of the picture compression process.

*data_unit_patternsstr

Force the generated sequence of data units to match a specified regular expression. For example, "(. padding_data)+ end_of_sequence" will force a padding data unit to be inserted between each data unit. See the vc2_conformance.symbol_re module for details of the regular expression format.

A sequence of data units matching all specified patterns while meeting the requirements of the VC-2 standard will be generated. If this is not possible, vc2_conformance.encoder.exceptions.IncompatibleLevelAndDataUnitError will be raised.

minimum_qindexint or [int, …]

Keyword-only argument. Default 0. Specifies the minimum quantization index to be used for all picture slices. If a list is provided, specifies the minimum quantization index separately for each picture.

This option may be used by test cases where a particular (very high) quantization index must be used. Note that the encoder may still use larger quantization indices if a set of transform coefficients still do not fit into a slice so the caller must check that this has not occurred.

Must be 0 for lossless coding modes.

minimum_slice_size_scalerint

Keyword-only argument. Default 1. Specifies the minimum slice size scaler to use.

For almost all sensible coding modes, the slice_size_scaler can be set to ‘1’ – and this encoder will do so if possible. To facilitate the production of test cases verifying higher values are supported, this option may be used to pick a larger value. The encoder may still use larger slice_size_scaler values if this is necessary, however.

Only has an effect on high quality profile coding modes, will be ignored for the low delay profile modes.

Returns
sequencevc2_conformance.bitstream.Sequence

The VC-2 bitstream sequence. This may be serialised by encapsulating it in a vc2_conformance.bitstream.Stream and serialising it with autofill_and_serialise_stream().

Raises
UnsatisfiableCodecFeaturesError

Raised if a sequence could not be generated according to the requirements given.

Level constraints

For the most part, all of the parameters which could be restricted by a VC-2 level are chosen in the supplied CodecFeatures. As such, choosing parameters which comply with the declared level is the responsibility of the caller (see comments above). However, some coding choices restricted by levels are left up to this encoder, such as how video parameters are coded in a sequence header. In these cases, the encoder makes choices which comply with the supplied level, a process which may require a constraint solving procedure.

In principle level constaints, as expressed by constraints tables (see vc2_conformance.constraint_table and vc2_conformance.level_constraints), could require a full global constraint solver to resolve. Fortunately, all existing VC-2 levels are specified such that, once the level (and a few other parameters) have been defined, almost all constrained parameters are independent meaning that global constraint solving is not required. The only case where constraint dependencies exist are the parameters relating to sequence headers. As a consequence the sequence_header generation module uses a simple constraint solver internally.

Note

The constraint parameter independence property of the VC-2 levels mentioned above is essential for the encoder to generate level-conforming bitstreams. A test in tests/encoder/test_level_constraints_assumptions.py is provided which will fail should a future VC-2 level not have this property. See the detailed documentation at the top of this file for a more thorough introduction and discussion of this topic.