vc2_conformance.bitstream: Bitstream manipulation module

The vc2_conformance.bitstream module implements facilities for deserialising, displaying, manipulating and serialising VC-2 bitstreams, including non-conformant streams, at a low-level.

This documentation begins with an overview of how bitstreams can be serialised, deserialised and represented as Python data structures using this module. This is followed by an in-depth description of how the serialiser and deserialisers work internally.

How the serialiser/deserialiser module is used

This module is used by various parts of the VC-2 conformance software, for example:

• The vc2-bitstream-viewer utility uses this module to produce human readable, hierarchical descriptions of bitstreams.

• The test case generators in vc2_conformance.test_cases use this module to manipulate bitstreams, for example by tweaking values or filling padding bits with specific data.

• The VC-2 encoder in vc2_conformance.encoder produces deserialised bitstreams directly for later serialisation by this module.

• The conformance software’s own test suite makes extensive use of this module.

Note

This module is not used by the bitstream validator (vc2_conformance.decoder) which instead operates directly on the binary bitstream instead.

This module consists of two main parts: the serdes framework for building serialisers and deserialisers and a serialiser/deserialiser for VC-2.

The serdes framework allows serialisers and deserialisers to be constructed directly from the VC-2 pseudocode, ensuring a high chance of correctness.

The VC-2 serialiser/deserialiser, implemented using the serdes framework defines a series of Python data structures which may be used to describe a VC-2 bitstream.

Quick-start guide

Before diving into the details, we’ll briefly give few quick examples which illustrate how this module is used below. We’ll show how a small bitstream can be explicitly described as a Python data structure, serialised and then deserialised again. We’ll skim over the details (and ignore a number of important but minor features) in the process.

Deserialised bitstream data structures

A VC-2 bitstream can be described hierarchically as a series of dictionaries and lists. For example, the following structure describes a minimal VC-2 bitstream containing just a single end-of-sequence data unit:

>>> from bitarray import bitarray

>>> # A minimal bitstream...
>>> bitstream_description = {
...     # ...consisting of a single sequence...
...     "sequences": [
...         {
...             # ...with a single data unit...
...             "data_units": [
...                 {
...                     # ...which is an end-of-sequence data unit
...                     "parse_info": {
...                         "padding": bitarray(),  # No byte alignment bits
...                         "parse_info_prefix": 0x42424344,
...                         "parse_code": 0x10,  # End of sequence
...                         "next_parse_offset": 0,
...                         "previous_parse_offset": 0,
...                     },
...                 },
...             ],
...         },
...     ],
... }

To make this somewhat clearer and more robust, a set of fixeddicts are provided which may be used instead of bare Python dictionaries. The vc2_data_tables package also includes many helpful constant definitions. Together, these make it easier to see what’s going on while also eliminating simple mistakes like misspelling a field name.

Note

fixeddicts are subclasses of Python’s native dict type with the following extra features:

• They allow only a permitted set of key names to be used.

• The __str__ implementation produces an easier to read pretty-printed format.

Using these types, our example now looks like:

>>> from vc2_data_tables import PARSE_INFO_PREFIX, ParseCodes
>>> from vc2_conformance.bitstream import Stream, Sequence, DataUnit, ParseInfo

>>> bitstream_description = Stream(
...     sequences=[
...         Sequence(
...             data_units=[
...                 DataUnit(
...                     parse_info=ParseInfo(
...                         padding=bitarray(),  # No byte alignment bits
...                         parse_info_prefix=PARSE_INFO_PREFIX,
...                         parse_code=ParseCodes.end_of_sequence,
...                         next_parse_offset=0,
...                         previous_parse_offset=0,
...                     ),
...                 ),
...             ],
...         ),
...     ],
... )

See Deserialised VC-2 bitstream data types for details of the expected hierarchy of a deserialised bitstream (and the fixeddict dictionary types provided).

Serialising bitstreams

To serialise our bitstream into binary form, we can use the following (which we’ll unpick afterwards):

>>> from vc2_conformance.bitstream import BitstreamWriter, Serialiser, parse_stream
>>> from vc2_conformance.pseudocode import State

>>> with open("/path/to/bitstream.vc2", "wb") as f:
...     with Serialiser(BitstreamWriter(f), bitstream_description) as ser:
...         parse_stream(ser, State())

In the example above, parse_stream is (a special version of) the parse_stream VC-2 pseudocode function provided by the vc2_conformance.bitstream module. This pseudocode function would normally decode a VC-2 stream (as per the VC-2 specification), however this modified version may be used to serialise (or deserialise) a bitstream. The modified function takes an extra first argument, a Serialiser in this case, which it will use to produce the serialised bitstream.

The Serialiser class takes two arguments in this example: a BitstreamWriter and the data structure to serialise (bitstream_description in our example).

The BitstreamWriter is a wrapper for file-like objects which provides additional bitwise I/O operations used during serialisation.

The second argument to Serialiser is the deserialised data structure to be serialised. This may be an ordinary Python dicts or a fixeddict.

Note

It is possible to serialise (or deserialise) components of a bitstream in isolation by using other pseudocode functions in place of parse_stream. In this case, the data structure provided to the Serialiser must match the shape expected by the modified pseudocode function used. See Deserialised VC-2 bitstream data types for an enumeration of the pseudocode functions available and the expected data structure.

Autofilling bitstream values

In the example above we explicitly spelt out every field in the bitstream – including the empty padding field! If we had omitted this field, the serialiser will produce an error because it wouldn’t know what padding bits we wanted it to use in the stream. However, often we don’t care about details such as these and so the Serialiser can optionally ‘autofill’ certain values which weren’t given in the deserialised data structure.

The Serialiser class takes an optional third argument which it uses to autofill missing values. A sensible set of a autofill values is provided in vc2_conformance.bitstream.vc2_default_values allowing us to rewrite our example like so:

>>> from vc2_conformance.bitstream import vc2_default_values

>>> concise_bitstream_description = Stream(
...     sequences=[
...         Sequence(
...             data_units=[
...                 DataUnit(
...                     parse_info=ParseInfo(
...                         parse_code=ParseCodes.end_of_sequence,
...                         next_parse_offset=0,
...                         previous_parse_offset=0,
...                     ),
...                 ),
...             ],
...         ),
...     ],
... )

>>> with open("/path/to/bitstream.vc2", "wb") as f:
...     with Serialiser(
...         BitstreamWriter(f),
...         concise_bitstream_description,
...         vc2_default_values,
...     ) as ser:
...         parse_stream(ser, State())

This time we were able to omit the byte-alignment padding value and parse info prefix which the serialiser autofilled with zeros and 0x42424344 respectively.

The default values for all fields are given in Deserialised VC-2 bitstream data types.

Unfortunately, when using the mechanism above, autofill values are not provided for every field in a bitstream (or for fields listed as autofilled with <AUTO> in the documentation). For instance, picture numbers and parse offset values must still be calculated and specified explicitly. For a more complete bitstream autofill solution the vc2_conformance.bitstream.autofill_and_serialise_stream() utility function is provided.

The autofill_and_serialise_stream() function can autofill most values including picture numbers, and parse offset fields (which marked as <AUTO> in the documentation). It also provides a more concise wrapper around the serialisation process.

Using autofill_and_serialise_stream() our example now becomes:

>>> from vc2_conformance.bitstream import autofill_and_serialise_stream

>>> very_concise_bitstream_description = Stream(
...     sequences=[
...         Sequence(
...             data_units=[
...                 DataUnit(
...                     parse_info=ParseInfo(
...                         parse_code=ParseCodes.end_of_sequence,
...                     ),
...                 ),
...             ],
...         ),
...     ],
... )

>>> with open("/path/to/bitstream.vc2", "wb") as f:
...     autofill_and_serialise_stream(f, very_concise_bitstream_description)

Notice that this time we could omit all but the parse_code field.

Note

The autofill_and_serialise_stream() function only supports serialisation of entire Streams and cannot be used to serialise smaller pieces of a bitstream in isolation.

Deserialising bitstreams

To deserialise a bitstream again, the process is similar:

>>> from vc2_conformance.bitstream import BitstreamReader, Deserialiser

>>> with open("/tmp/bitstream.vc2", "rb") as f:
...     with Deserialiser(BitstreamReader(f)) as des:
...         parse_stream(des, State())
>>> deserialised_bitstream = des.context

This time, we pass a Deserialiser (which takes a BitstreamReader as argument) into parse_stream. The deserialised bitstream is placed into des.context as a hierarchy of fixeddicts. We can then print or interact with the deserialised data structure just like any other Python object:

>>> # NB: Fixeddicts produce pretty-printed output when printed!
>>> print(deserialised_bitstream)
Stream:
  sequences:
    0: Sequence:
      data_units:
        0: DataUnit:
          parse_info: ParseInfo:
            padding: 0b
            parse_info_prefix: Correct (0x42424344)
            parse_code: end_of_sequence (0x10)
            next_parse_offset: 0
            previous_parse_offset: 0

>>> data_unit = deserialised_bitstream["sequences"][0]["data_units"][0]
>>> print(data_unit["parse_info"]["parse_code"])
16

Deserialised VC-2 bitstream data types

Deserialised VC-2 bitstreams are described by a hierarchy of fixeddicts, exported in vc2_conformance.bitstream. Each fixeddict represents the data read by a particular VC-2 pseudocode function. Special implementations of these functions are provided in vc2_conformance.bitstream which may be used to serialise and deserialise VC-2 bitstreams (or individual parts thereof).

Type	Pseudocode function
Stream	parse_stream
Sequence	parse_sequence
DataUnit
ParseInfo	parse_info
SequenceHeader	sequence_header
ParseParameters	parse_parameters
SourceParameters	source_parameters
FrameSize	frame_size
ColorDiffSamplingFormat	color_diff_sampling_format
ScanFormat	scan_format
FrameRate	frame_rate
PixelAspectRatio	pixel_aspect_ratio
CleanArea	clean_area
SignalRange	signal_range
ColorSpec	color_spec
ColorPrimaries	color_primaries
ColorMatrix	color_matrix
TransferFunction	transfer_function
PictureParse	picture_parse
PictureHeader	picture_header
WaveletTransform	wavelet_transform
TransformParameters	transform_parameters
ExtendedTransformParameters	extended_transform_parameters
SliceParameters	slice_parameters
QuantMatrix	quant_matrix
TransformData	transform_data
LDSlice	ld_slice
HQSlice	hq_slice
FragmentParse	fragment_parse
FragmentHeader	fragment_header
TransformParameters	transform_parameters
ExtendedTransformParameters	extended_transform_parameters
SliceParameters	slice_parameters
QuantMatrix	quant_matrix
FragmentData	fragment_data
LDSlice	ld_slice
HQSlice	hq_slice
AuxiliaryData	auxiliary_data
Padding	padding

fixeddict Stream

(10.3) A VC-2 stream.

Keys

sequences[Sequence, …]

fixeddict Sequence

(10.4.1) A VC-2 sequence.

Keys

data_units[DataUnit, …]

_stateState

Computed value. The State object being populated by the parser.

fixeddict DataUnit

A data unit (e.g. sequence header or picture) and its associated parse info. Based on the values read by parse_sequence() (10.4.1) in each iteration.

Keys

parse_infoParseInfo

sequence_headerSequenceHeader

picture_parsePictureParse

fragment_parseFragmentParse

auxiliary_dataAuxiliaryData

paddingPadding

fixeddict ParseInfo

(10.5.1) Parse info header defined by parse_info().

Keys

paddingbitarray (autofilled with bitarray())

Byte alignment padding bits.

parse_info_prefixint (autofilled with 1111638852)

parse_codeParseCodes (autofilled with <ParseCodes.end_of_sequence: 16>)

next_parse_offsetint (autofilled with <AUTO>)

previous_parse_offsetint (autofilled with <AUTO>)

_offsetint

Computed value. The byte offset of the start of this parse_info block in the bitstream.

fixeddict SequenceHeader

(11.1) Sequence header defined by sequence_header().

Keys

parse_parametersParseParameters

base_video_formatBaseVideoFormats (autofilled with <BaseVideoFormats.custom_format: 0>)

video_parametersSourceParameters

picture_coding_modePictureCodingModes (autofilled with <PictureCodingModes.pictures_are_frames: 0>)

fixeddict ParseParameters

(11.2.1) Sequence header defined by parse_parameters().

Keys

major_versionint (autofilled with <AUTO>)

minor_versionint (autofilled with 0)

profileProfiles (autofilled with <Profiles.high_quality: 3>)

levelLevels (autofilled with <Levels.unconstrained: 0>)

fixeddict SourceParameters

(11.4.1) Video format overrides defined by source_parameters().

Keys

frame_sizeFrameSize

color_diff_sampling_formatColorDiffSamplingFormat

scan_formatScanFormat

frame_rateFrameRate

pixel_aspect_ratioPixelAspectRatio

clean_areaCleanArea

signal_rangeSignalRange

color_specColorSpec

fixeddict FrameSize

(11.4.3) Frame size override defined by frame_size().

Keys

custom_dimensions_flagbool (autofilled with False)

frame_widthint (autofilled with 1)

frame_heightint (autofilled with 1)

fixeddict ColorDiffSamplingFormat

(11.4.4) Color-difference sampling override defined by color_diff_sampling_format().

Keys

custom_color_diff_format_flagbool (autofilled with False)

color_diff_format_indexColorDifferenceSamplingFormats (autofilled with <ColorDifferenceSamplingFormats.color_4_4_4: 0>)

fixeddict ScanFormat

(11.4.5) Scan format override defined by scan_format().

Keys

custom_scan_format_flagbool (autofilled with False)

source_samplingSourceSamplingModes (autofilled with <SourceSamplingModes.progressive: 0>)

fixeddict FrameRate

(11.4.6) Frame-rate override defined by frame_rate().

Keys

custom_frame_rate_flagbool (autofilled with False)

indexPresetFrameRates (autofilled with <PresetFrameRates.fps_25: 3>)

frame_rate_numerint (autofilled with 25)

frame_rate_denomint (autofilled with 1)

fixeddict PixelAspectRatio

(11.4.7) Pixel aspect ratio override defined by pixel_aspect_ratio() (errata: also listed as aspect_ratio() in some parts of the spec).

Keys

custom_pixel_aspect_ratio_flagbool (autofilled with False)

indexPresetPixelAspectRatios (autofilled with <PresetPixelAspectRatios.ratio_1_1: 1>)

pixel_aspect_ratio_numerint (autofilled with 1)

pixel_aspect_ratio_denomint (autofilled with 1)

fixeddict CleanArea

(11.4.8) Clean areas override defined by clean_area().

Keys

custom_clean_area_flagbool (autofilled with False)

clean_widthint (autofilled with 1)

clean_heightint (autofilled with 1)

left_offsetint (autofilled with 0)

top_offsetint (autofilled with 0)

fixeddict SignalRange

(11.4.9) Signal range override defined by signal_range().

Keys

custom_signal_range_flagbool (autofilled with False)

indexPresetSignalRanges (autofilled with <PresetSignalRanges.video_8bit_full_range: 1>)

luma_offsetint (autofilled with 0)

luma_excursionint (autofilled with 1)

color_diff_offsetint (autofilled with 0)

color_diff_excursionint (autofilled with 1)

fixeddict ColorSpec

(11.4.10.1) Color specification override defined by color_spec().

Keys

custom_color_spec_flagbool (autofilled with False)

indexPresetColorSpecs (autofilled with <PresetColorSpecs.hdtv: 3>)

color_primariesColorPrimaries

color_matrixColorMatrix

transfer_functionTransferFunction

fixeddict ColorPrimaries

(11.4.10.2) Color primaries override defined by color_primaries().

Keys

custom_color_primaries_flagbool (autofilled with False)

indexPresetColorPrimaries (autofilled with <PresetColorPrimaries.hdtv: 0>)

fixeddict ColorMatrix

(11.4.10.3) Color matrix override defined by color_matrix().

Keys

custom_color_matrix_flagbool (autofilled with False)

indexPresetColorMatrices (autofilled with <PresetColorMatrices.hdtv: 0>)

fixeddict TransferFunction

(11.4.10.4) Transfer function override defined by transfer_function().

Keys

custom_transfer_function_flagbool (autofilled with False)

indexPresetTransferFunctions (autofilled with <PresetTransferFunctions.tv_gamma: 0>)

fixeddict PictureParse

(12.1) A picture data unit defined by picture_parse()

Keys

padding1bitarray (autofilled with bitarray())

Picture header byte alignment padding bits.

picture_headerPictureHeader

padding2bitarray (autofilled with bitarray())

Wavelet transform byte alignment padding bits.

wavelet_transformWaveletTransform

fixeddict PictureHeader

(12.2) Picture header information defined by picture_header().

Keys

picture_numberint (autofilled with <AUTO>)

fixeddict WaveletTransform

(12.3) Wavelet parameters and coefficients defined by wavelet_transform().

Keys

transform_parametersTransformParameters

paddingbitarray (autofilled with bitarray())

Byte alignment padding bits.

transform_dataTransformData

fixeddict TransformParameters

(12.4.1) Wavelet transform parameters defined by transform_parameters().

Keys

wavelet_indexWaveletFilters (autofilled with <WaveletFilters.haar_with_shift: 4>)

dwt_depthint (autofilled with 0)

extended_transform_parametersExtendedTransformParameters

slice_parametersSliceParameters

quant_matrixQuantMatrix

fixeddict ExtendedTransformParameters

(12.4.4.1) Extended (horizontal-only) wavelet transform parameters defined by extended_transform_parameters().

Keys

asym_transform_index_flagbool (autofilled with False)

wavelet_index_hoWaveletFilters (autofilled with <WaveletFilters.haar_with_shift: 4>)

asym_transform_flagbool (autofilled with False)

dwt_depth_hoint (autofilled with 0)

fixeddict SliceParameters

(12.4.5.2) Slice dimension parameters defined by slice_parameters().

Keys

slices_xint (autofilled with 1)

slices_yint (autofilled with 1)

slice_bytes_numeratorint (autofilled with 1)

slice_bytes_denominatorint (autofilled with 1)

slice_prefix_bytesint (autofilled with 0)

slice_size_scalerint (autofilled with 1)

fixeddict QuantMatrix

(12.4.5.3) Custom quantisation matrix override defined by quant_matrix().

Keys

custom_quant_matrixbool (autofilled with False)

quant_matrix[int, …] (autofilled with 0)

Quantization matrix values in bitstream order.

fixeddict TransformData

(13.5.2) Transform coefficient data slices read by transform_data().

Keys

ld_slices[LDSlice, …]

hq_slices[HQSlice, …]

_stateState

Computed value. A copy of the State dictionary held when processing this transform data. May be used to work out how the deserialised values correspond to transform components within the slices above.

fixeddict LDSlice

(13.5.3.1) The data associated with a single low-delay slice, defined by ld_slice().

Keys

qindexint (autofilled with 0)

slice_y_lengthint (autofilled with 0)

y_transform[int, …] (autofilled with 0)

Slice luma transform coefficients in bitstream order.

c_transform[int, …] (autofilled with 0)

Slice interleaved colordifference transform coefficients in bitstream order.

y_block_paddingbitarray (autofilled with bitarray())

Unused bits from y_transform bounded block.

c_block_paddingbitarray (autofilled with bitarray())

Unused bits from c_transform bounded block.

_sxint

Computed value. Slice coordinates.

_syint

Computed value. Slice coordinates.

fixeddict HQSlice

(13.5.4) The data associated with a single high-quality slice, defined by hq_slice().

Keys

prefix_bytesbytes (autofilled with b’’)

qindexint (autofilled with 0)

slice_y_lengthint (autofilled with 0)

slice_c1_lengthint (autofilled with 0)

slice_c2_lengthint (autofilled with 0)

y_transform[int, …] (autofilled with 0)

Slice luma transform coefficients in bitstream order.

c1_transform[int, …] (autofilled with 0)

Slice color difference 1 transform coefficients in bitstream order.

c2_transform[int, …] (autofilled with 0)

Slice color difference 2 transform coefficients in bitstream order.

y_block_paddingbitarray (autofilled with bitarray())

Unused bits in y_transform bounded block

c1_block_paddingbitarray (autofilled with bitarray())

Unused bits in c1_transform bounded block

c2_block_paddingbitarray (autofilled with bitarray())

Unused bits in c2_transform bounded block

_sxint

Computed value. Slice coordinates.

_syint

Computed value. Slice coordinates.

fixeddict FragmentParse

(14.1) A fragment data unit defined by fragment_parse() containing part of a picture.

Keys

fragment_headerFragmentHeader

transform_parametersTransformParameters

fragment_dataFragmentData

fixeddict FragmentHeader

(14.2) Fragment header defined by fragment_header().

Keys

picture_numberint (autofilled with <AUTO>)

fragment_data_lengthint (autofilled with 0)

fragment_slice_countint (autofilled with 0)

fragment_x_offsetint (autofilled with 0)

fragment_y_offsetint (autofilled with 0)

fixeddict FragmentData

(14.4) Transform coefficient data slices read by fragment_data().

Keys

ld_slices[LDSlice, …]

hq_slices[HQSlice, …]

_stateState

Computed value. A copy of the State dictionary held when processing this fragment data. May be used to work out how the deserialised values correspond to transform components within the slices above.

fixeddict AuxiliaryData

(10.4.4) Auxiliary data block (as per auxiliary_data()).

Keys

bytesbytes (autofilled with b’’)

fixeddict Padding

(10.4.5) Padding data block (as per padding()).

Keys

bytesbytes (autofilled with b’’)

serdes: A serialiser/deserialiser framework

The vc2_conformance.bitstream.serdes module provides a framework for transforming a set of functions designed to process a bitstream (e.g. the VC-2 specification’s pseudocode) into general-purpose bitstream serialisers, deserialisers and analysers.

The following sections introduce the design and operation of this module in detail.

A basic bitstream serialiser

The VC-2 specification describes the bitstream and decoding process in a series of pseudocode functions such as the following:

frame_size(video_parameters):
    custom_dimensions_flag = read_bool()
    if(custom_dimensions_flag == True)
        video_parameters[frame_width] = read_uint()
        video_parameters[frame_height] = read_uint()

To see how this definition might be transformed into a general purpose bitstream serialiser we must transform this definition of a program which reads a VC-2 bitstream into one which writes one.

We start by replacing all of the read_* functions with equivalent write_* functions (which we define here as returning the value that they write):

frame_size(video_parameters):
    custom_dimensions_flag = write_bool(???)
    if(custom_dimensions_flag == True)
        video_parameters[frame_width] = write_uint(???)
        video_parameters[frame_height] = write_uint(???)

Next we need to define what values we’d like writing by replacing the ??? placeholders with a suitable global variables like so:

new_custom_dimensions_flag = True
new_frame_width = 1920
new_frame_height = 1080

frame_size(video_parameters):
    custom_dimensions_flag = write_bool(new_custom_dimensions_flag)
    if(custom_dimensions_flag == True)
        video_parameters[frame_width] = write_uint(new_frame_width)
        video_parameters[frame_height] = write_uint(new_frame_height)

We have now transformed the VC-2 pseudocode bitstream reader function into a writer function. What’s more, by just changing the values of global variables we created it is possible to use this function as a general-purpose bitstream serialiser.

A basic deserialiser

Unfortunately, the original bitstream reader pseudocode from the VC-2 specification is not quite usable as a general-purpose bitstream deserialiser:

• The reader does not capture every value read from the bitstream in a variable we can later examine (e.g. the custom_dimensions_flag is kept in a local variable and not returned).

• The values which are captured are stored in a structure designed to aid decoding and not necessarily to faithfully describe a bitstream.

Lets create a new version of the reader function which overcomes these limitations. We redefine the read_* functions to take an additional argument naming a global variable where the read values will be stored, in addition to being returned, giving the following pseudocode:

read_custom_dimensions_flag = None
read_frame_width = None
read_frame_height = None

frame_size(video_parameters):
    custom_dimensions_flag = read_bool(read_custom_dimensions_flag)
    if(custom_dimensions_flag == True)
        video_parameters[frame_width] = read_uint(read_frame_width)
        video_parameters[frame_height] = read_uint(read_frame_height)

This small change ensures that every value read from the bitstream is captured in a global variable which we can later examine and which is orthogonal to whatever data structures the VC-2 pseudocode might otherwise use.

An introduction to the SerDes interface

The similarities between the transformations required to turn the VC-2 pseudocode into general purpose serialisers and deserialisers should be fairly clear. In fact, the only difference between the two is that in one the functions are called read_* and in the other they’re called write_*. In both cases, the read_* and write_* functions take essentially the same arguments: a name of a global variable.

This module defines the SerDes interface which can be used by the VC-2 pseudocode specifications to drive both bitstream serialisation and deserialisation. To use it, we replace the read_* or write_* calls with serdes.* calls.

Translating the frame_size function into valid Python and taking a SerDes instance as an argument we arrive at the following code:

def frame_size(serdes, video_parameters):
    custom_dimensions_flag = serdes.bool("custom_dimensions_flag")
    if(custom_dimensions_flag == True)
        video_parameters["frame_width"] = serdes.uint("frame_width")
        video_parameters["frame_height"] = serdes.uint("frame_height")

To deserialise (read) a bitstream we use the Deserialiser implementation of SerDes like so:

>>> from vc2_conformance.bitstream import BitstreamReader, Deserialiser
>>> reader = BitstreamReader(open("frame_size_snippet.bin", "rb"))

>>> with Deserialiser(reader) as des:
...     frame_size(des, {})
>>> des.context
{"custom_dimensions_flag": True, "frame_width": 1920, "frame_height": 1080}

The SerDes.context property is a dict which contains each of the values read from the bitstream (named as per the calls to the various SerDes methods).

In the nomenclature of this module, this context dictionary holds values for each of the target names specified by SerDes.bool(), SerDes.uint() etc.

Values to be serialised should be structured into a context dictionary of similar shape and passed to a Serialiser:

>>> from vc2_conformance.bitstream import BitstreamWriter, Serialiser
>>> writer = BitstreamWriter(open("frame_size_snippet.bin", "wb"))

>>> context = {"custom_dimensions_flag": True, "frame_width": 1920, "frame_height": 1080}
>>> with Serialiser(writer, context) as ser:
...     frame_size(ser, {})

In this example a bitstream containing a ‘1’ followed by the variable-length integers ‘1920’ and ‘1080’ would be written to the bitstream.

Verification

The SerDes implementations perform various ‘sanity checks’ during serialisation and deserialisation to ensure that the values passed in or returned have a 1:1 correspondence with values in the bitstream.

• When values are read during deserialisation, Deserialiser checks that names are not re-used, guaranteeing that if a value appears in the bitstream it also appears in the output context dictionary (and are not later overwritten).

• When values are written during serialisation, Serialiser checks that every value in the context dictionary is used exactly once, ensuring that every value provided is represented in the bitstream.

• During serialisation, values are also checked to ensure they can be correctly represented by the bitstream encoding. For example, providing a negative value to uint() will fail.

Representing hierarchy

The VC-2 bitstream does not represent a flat collection of values but rather a hierarchy. The SerDes interface provides additional facilities to allow this structure to be recreated in the deserialised representation, making it easier to inspect and describes bitstreams in their deserialised form.

For example, in the VC-2 specification, the source_parameters function (11.4) is defined by a series of functions which each read the values relating to a particular video feature such as the frame_size function we’ve seen above. To collect together related values we can use SerDes.subcontext() to create nested context dictionaries:

def source_parameters(serdes):
    video_parameters = {}
    with serdes.subcontext("frame_size"):
        frame_size(serdes, video_parameters)
    with serdes.subcontext("color_diff_sampling_format"):
        color_diff_sampling_format(serdes, video_parameters)
    # ...
    return video_parameters

This results in a nested dictionary structure:

>>> with Deserialiser(reader) as des:
...     video_parameters = source_parameters(des)

>>> from pprint import pprint
>>> pprint(des.context)
{
    "frame_size": {
        "custom_dimensions_flag": True,
        "frame_width": 1920,
        "frame_height": 1080,
    },
    "color_diff_sampling_format": {
        "custom_color_diff_format_flag": False,
    },
    # ...
}

When used with vc2_conformance.fixeddict, SerDes also makes it possible to define custom dictionary types for each part of the hierarchy using the context_type() decorator. Benefits include:

• Improved ‘pretty-printed’ string representations.

• Additional checks that unexpected values are not used accidentally in the bitstream.

For example, here’s how the parse_info header (10.5.1) might be represented:

from vc2_conformance.fixeddict import fixeddict, Entry
from vc2_conformance.formatters import Hex
from vc2_data_tables import ParseCodes, PARSE_INFO_PREFIX
ParseInfo = fixeddict(
    "ParseInfo",
    Entry("parse_info_prefix", formatter=Hex(8)),
    Entry("parse_code", enum=ParseCodes),
    Entry("next_parse_offset"),
    Entry("previous_parse_offset"),
)

@context_type(ParseInfo)
def parse_info(serdes, state):
    serdes.nbits(4*8, "parse_info_prefix")
    state["parse_code"] = serdes.nbits(8, "parse_code")
    state["next_parse_offset"] = serdes.nbits(32, "next_parse_offset")
    state["previous_parse_offset"] = serdes.nbits(32, "previous_parse_offset")

Using the above we can quickly create structures ready for serialisation:

>>> context = ParseInfo(
...     parse_info_prefix=PARSE_INFO_PREFIX,
...     parse_code=ParseCodes.end_of_sequence,
...     next_parse_offset=0,
...     previous_parse_offset=1234,
... )
>>> with Deserialiser(writer, context) as des:
...     parse_info(des, {})

We also benefit from improved string formatting when deserialising values:

>>> with Deserialiser(reader) as des:
...     parse_info(des, {})
>>> str(des.context)
ParseInfo:
  parse_info_prefix: 0x42424344
  parse_code: end_of_sequence (0x10)
  next_parse_offset: 0
  previous_parse_offset: 1234

Representing arrays

The VC-2 bitstream format includes a number of array-like fields, for example arrays of transform coefficients within slices. Rather than defining unique names for every array value, SerDes allows values to be declared as lists. For example:

def list_example(serdes):
    serdes.declare_list("three_values")
    serdes.uint("three_values")
    serdes.uint("three_values")
    serdes.uint("three_values")

When deserialising, the result will look like:

>>> with Deserialiser(reader) as des:
...     list_example(des)
>>> des.context
{"three_values": [100, 200, 300]}

Likewise, when serialising, a list of values (of the correct length) should also be provided:

>>> context = {"three_values": [10, 20, 30]}
>>> with Deserialiser(writer, context) as des:
...     list_example(des)

As usual, the SerDes classes will verify that the correct number of values is present and will throw exceptions when too many or too few are provided.

Computed values

In some circumstances, when interpreting a deserialised bitstream it may be necessary to know information computed by an earlier part of the bitstream. For example, the dimensions of a slice depend on numerous video formatting options. To avoid error-prone reimplementation of these calculations it is possible to use SerDes.computed_value() to add values to the context dictionary which do not appear in the bitstream. For example:

def ld_slice(serdes, state, sx, sy):
    serdes.computed_value("_slices_x", state["slices_x"])
    serdes.computed_value("_slices_y", state["slices_y"])
    # ...

The computed value will be set in the context dictionary regardless of whether serialisation or deserialisation is taking place and any existing value is always ignored.

Note

It is recommended that by convention computed value target names are prefixed or suffixed with an underscore.

Default values during serialisation

As discussed above, the default behaviour of the Serialiser is to require that every value in the bitstream is provided in the context dictionary to make it explicit what is being serialised. In certain cases, however, it may be desirable for certain values to be filled in automatically. For example:

• For pre-filling constants like the parse_info prefix.

• For use in unit tests where only certain bitstream fields’ values are of importance (and assigning defaults for the remainder makes the code clearer).

• For providing default (e.g. zero) values for padding fields

To facilitate this, the Serialiser class may be passed a default value lookup like so:

>>> default_values = {
...     ParseInfo: {
...         "parse_info_prefix": PARSE_INFO_PREFIX,
...         "parse_code": ParseCodes.end_of_sequence,
...         "next_parse_offset": 0,
...         "previous_parse_offset": 0,
...     },
... }

>>> writer = BitstreamWriter(open("frame_size_snippet.bin", "wb"))
>>> context = ParseInfo(
...     parse_code=ParseCodes.end_of_sequence,
...     previous_parse_offset=123,
... )
>>> with Serialiser(writer, context, default_values=default_values) as ser:
...     parse_info(ser, {})

The default_values lookup should provide a separate set of default values for each context dictionary type. See vc2_conformance.bitstream.vc2_fixeddicts.fixeddict_default_values for a complete example.

For arrays/lists of values, the default value provided will be usd to populate array elements and not to provide a default for the list as a whole.

Where a default value is not found in the lookup, a :py;exc:KeyError will be thrown as usual. This behaviour allows a partial set of default values to be provided (e.g. providing defaults only for padding values) while still validating that the provided input is correct.

API

class SerDes(io, context=None)

The base serialiser/deserialiser interface and implementation.

This base implementation includes all but the value writing/reading features of the serialisation and deserialisation process.

Attributes

ioBitstreamReader or BitstreamWriter

The I/O interface in use.

contextdict or None

Get the top-level context dictionary.

cur_contextdict or None

The context dictionary currently being populated.

bool(target)

Reads or writes a boolean (single bit) in a bitstream (as per (A.3.2) read_bool()).

Parameters

targetstr

The target for the bit (as a bool).

Returns

valuebool

nbits(target, num_bits)

Reads or writes a fixed-length unsigned integer in a bitstream (as per (A.3.3) read_nbits()).

Parameters

targetstr

The target for the value (as an int).

num_bitsint

The number of bits in the value.

Returns

valueint

uint_lit(target, num_bytes)

Reads or writes a fixed-length unsigned integer in a bitstream (as per (A.3.4) read_uint_lit()). Not to be confused with uint().

Parameters

targetstr

The target for the value (as an int).

num_bytesint

The number of bytes in the value.

Returns

valueint

bitarray(target, num_bits)

Reads or writes a fixed-length string of bits from the bitstream as a bitarray.bitarray. This may be a more sensible type for holding unpredictably sized non-integer binary values such as padding bits.

Parameters

targetstr

The target for the value (as a bitarray.bitarray).

num_bitsint

The number of bits in the value.

Returns

valuebitarray.bitarray

bytes(target, num_bytes)

Reads or writes a fixed-length bytes string from the bitstream. This is a more convenient alternative to nbits() or bitarray() when large blocks of data are to be read but not treated as integers.

Parameters

targetstr

The target for the value (as a bytes).

num_bitsint

The number of bytes (not bits) in the value.

Returns

valuebytes

uint(target)

A variable-length, unsigned exp-golomb integer in a bitstream (as per (A.4.3) read_uint()).

Parameters

targetstr

The target for the value (as an int).

Returns

valueint

sint(target, num_bits)

A variable-length, signed exp-golomb integer in a bitstream (as per (A.4.4) read_sint()).

Parameters

targetstr

The target for the value (as an int).

Returns

valueint

byte_align(target)

Advance in the bitstream to the next whole byte boundary, if not already on one (as per (A.2.4) byte_align()).

Parameters

targetstr

The target for the padding bits (as a bitarray.bitarray).

bounded_block_begin(length)

Defines the start of a bounded block (as per (A.4.2)). Must be followed by a matching bounded_block_end().

See also: bounded_block().

Bits beyond the end of the block are always ‘1’. If a ‘0’ is written past the end of the block a ValueError will be thrown.

Parameters

lengthint

The length of the bounded block in bits

bounded_block_end(target)

Defines the end of a bounded block (as per (A.4.2)). Must be proceeded by a matching bounded_block_begin().

Parameters

targetstr

The target name for any unused bits (as a bitarray.bitarray).

bounded_block(target, length)

A context manager defining a bounded block (as per (A.4.2)).

See also: bounded_block_begin().

Example usage:

with serdes.bounded_block("unused_bits", 100):
    # ...

Parameters

targetstr

The target name for any unused bits (as a bitarray.bitarray).

lengthint

The length of the bounded block in bits

declare_list(target)

Declares that the specified target should be treated as a list. Whenever this target is used in the future, values will be read/written sequentially from the list.

This method has no impact on the bitstream.

Parameters

targetstr

The target name to be declared as a list.

set_context_type(context_type)

Set (or change) the type of the current context dictionary.

This method has no impact on the bitstream.

Parameters

context_typedict-like type

The desired type. If the context is already of the required type, no change will be made. If the context is currently of a different type, it will be passed to the context_type constructor and the new type used in its place.

subcontext_enter(target)

Creates and/or enters a context dictionary within the specified target of the current context dictionary. Must be followed later by a matching subcontext_leave().

Parameters

targetstr

The name of the target in the current context in which the new subcontext is/will be stored.

subcontext_leave()

Leaves the current nested context dictionary entered by subcontext_enter(). Verifies that the closed dictionary has no unused entries, throwing an appropriate exception if not.

subcontext(target)

A Python context manager alternative to ;py:meth:subcontext_enter and ;py:meth:subcontext_leave.

Example usage:

>>> with serdes.subcontext("target"):
...     # ...

Exactly equivalent to:

>>> serdes.subcontext_enter("target"):
>>> # ...
>>> serdes.subcontext_leave():

(But without the possibility of forgetting the subcontext_leave() call).

Parameters

targetstr

The name of the target in the current context in which the new subcontext is/will be stored.

computed_value(target, value)

Places a value into the named target in the current context, without reading or writing anything into the bitstream. Any existing value in the context will be overwritten.

This operation should be used sparingly to embed additional information in a context dictionary which might be required to sensibly interpret its contents and which cannot be trivially computed from the context dictionary. For example, the number of transform coefficients in a coded picture depends on numerous computations and table lookups using earlier bitstream values.

Parameters

targetstr

The name of the target in the current context to store the value in.

valueany

The value to be stored.

is_target_complete(target)

Test whether a target in the current context is complete, i.e. has been fully used up. Returns True if so, False otherwise.

verify_complete()

Verify that all values in the current context have been used and that no bounded blocks or nested contexts have been left over.

Raises

UnusedTargetError

UnclosedNestedContextError

UnclosedBoundedBlockError

property context

Get the top-level context dictionary.

path(target=None)

Produce a ‘path’ describing the part of the bitstream the parser is currently processing.

If ‘target’ is None, only includes the path of the current nested context dictionary. If ‘target’ is a target name in the current target dictionary, the path to the last-used target will be included.

A path might look like:

['source_parameters', 'frame_size', 'frame_width']

describe_path(target=None)

Produce a human-readable description of the part of the bitstream the parser is currently processing.

If ‘target’ is None, prints only the path of the current nested context dictionary. If ‘target’ is a target name in the current target dictionary, this will be included in the string.

As a sample, a path might look like the following:

SequenceHeader['source_parameters']['frame_size']['frame_width']

class Serialiser(io, context=None, default_values={})

Bases: vc2_conformance.bitstream.serdes.SerDes

A bitstream serialiser which, given a populated context dictionary, writes the corresponding bitstream.

Parameters

ioBitstreamWriter

contextdict

class Deserialiser(io, context=None)

Bases: vc2_conformance.bitstream.serdes.SerDes

A bitstream deserialiser which creates a context dictionary based on a bitstream.

Parameters

ioBitstreamReader

class MonitoredSerialiser(monitor, *args, **kwargs)

Bases: vc2_conformance.bitstream.serdes.MonitoredMixin, vc2_conformance.bitstream.serdes.Serialiser

Like Serialiser but takes a ‘monitor’ function as the first constructor argument. This function will be called every time bitstream value has been serialised (written).

Parameters

monitorcallable(ser, target, value)

A function which will be called after every primitive I/O operation completes. This function is passed this MonitoredSerialiser instance and the target name and value of the target just serialised.

This function may be used to inform a user of the current progress of serialisation (e.g. using SerDes.describe_path() or SerDes.io) or to terminate serialisation early (by throwing an exception).

*args, **kwargssee Serialiser

class MonitoredDeserialiser(monitor, *args, **kwargs)

Bases: vc2_conformance.bitstream.serdes.MonitoredMixin, vc2_conformance.bitstream.serdes.Deserialiser

Like Deserialiser but takes a ‘monitor’ function as the first constructor argument. This function will be called every time bitstream value has been deserialised (read).

Parameters

monitorcallable(des, target, value)

A function which will be called after every primitive I/O operation completes. This function is passed this MonitoredDeserialiser instance and the target name and value of the target just serialised.

This function may be used to inform a user of the current progress of deserialisation (e.g. using SerDes.context(), SerDes.describe_path() or SerDes.io) or to terminate deserialisation early (by throwing an exception).

*args, **kwargssee Serialiser

context_type(dict_type)

Syntactic sugar. A decorator for SerDes which uses SerDes.set_context_type() to set the type of the current context dict:

Example usage:

@context_type(FrameSize)
def frame_size(serdes):
    # ...

Exactly equivalent to:

def frame_size(serdes):
    serdes.set_context_type(FrameSize)
    # ...

The wrapped function must take a SerDes as its first argument.

For introspection purposes, the wrapper function will be given a ‘context_type’ attribute holding the passed ‘dict_type’.

Low-level bitstream IO

The vc2_conformance.bitstream.io module contains low-level wrappers for file-like objects which facilitate bitwise read and write operations of the kinds used by VC-2’s bitstream format.

The BitstreamReader and BitstreamWriter classes provide equivalent methods for the various read_* pseudocode functions defined in the VC-2 specification, along with a few additional utility methods.

Note

These methods are designed to be ‘safe’ meaning that if out-of-range values are provided an error will be produced (rather than an unexpected value being written/read).

class BitstreamReader(file)

An open file which may be read one bit at a time.

When the end-of-file is encountered, reads will result in a EOFError.

is_end_of_stream()

Check if we’ve reached the EOF. (A.2.5)

tell()

Report the current bit-position within the stream.

Returns

(bytes, bits)

bytes is the offset of the current byte from the start of the stream. bits is the offset in the current byte (starting at 7 (MSB) and advancing towards 0 (LSB) as bits are read).

seek(bytes, bits=7)

Seek to a specific (absolute) position in the file.

Parameters

bytesint

The byte-offset from the start of the file.

bitsint

The bit offset into the specified byte to start reading from.

property bits_remaining

The number of bits left in the current bounded block.

None, if not in a bounded block. Otherwise, the number of unused bits remaining in the block. If negative, indicates the number of bits read past the end of the block.

bounded_block_begin(length)

Begin a bounded block of the specified length in bits.

bounded_block_end()

Ends the current bounded block. Returns the number of unused bits remaining, but does not read them or seek past them.

read_bit()

Read and return the next bit in the stream. (A.2.3) Reads ‘1’ for values past the end of file.

read_nbits(bits)

Read an ‘bits’-bit unsigned integer (like read_nbits (A.3.3)).

read_uint_lit(num_bytes)

Read a ‘num-bytes’ long integer (like read_uint_lit (A.3.4)).

read_bitarray(bits)

Read ‘bits’ bits returning the value as a bitarray.bitarray.

read_bytes(num_bytes)

Read a number of bytes returning a bytes string.

read_uint()

Read an unsigned exp-golomb code (like read_uint (A.4.3)) and return an integer.

read_sint()

Signed version of read_uint() (like read_sint (A.4.4)).

try_read_bitarray(bits)

Attempt to read the next ‘bits’ bits from the bitstream file, leaving any bounded blocks we might be in if necessary). May read fewer bits if the end-of-file is encountered (but will not throw a EOFError like other methods of this class).

Intended for the display of error messages (i.e. as the final use of a BitstreamReader instance) only since this method may (or may not) exit the current bounded block as a side effect.

class BitstreamWriter(file)

An open file which may be written one bit at a time.

is_end_of_stream()

Always True. (A.2.5)

Note

Strictly speaking this should return False when seeking to an earlier part of the stream however this behaviour is not implemented here for simplicity’s sake.

tell()

Report the current bit-position within the stream.

Returns

(bytes, bits)

bytes is the offset of the current byte from the start of the stream. bits is the offset in the current byte (starting at 7 (MSB) and advancing towards 0 (LSB) as bits are written).

seek(bytes, bits=7)

Seek to a specific (absolute) position in the file. Seeking to a given byte will overwrite any bits already set in that byte to 0.

Parameters

bytesint

The byte-offset from the start of the file.

bitsint

The bit offset into the specified byte to start writing to.

flush()

Ensure all bytes are committed to the file.

property bits_remaining

The number of bits left in the current bounded block.

None, if not in a bounded block. Otherwise, the number of unused bits remaining in the block. If negative, indicates the number of bits read past the end of the block.

bounded_block_begin(length)

Begin a bounded block of the specified length in bits.

bounded_block_end()

Ends the current bounded block. Returns the number of unused bits remaining, but does not write them or seek past them.

write_bit(value)

Write a bit into the bitstream. If in a bounded block, raises a ValueError if a ‘0’ is written beyond the end of the block.

write_nbits(bits, value)

Write an ‘bits’-bit integer. The complement of read_nbits (A.3.3).

Throws an OutOfRangeError if the value is too large to fit in the requested number of bits.

write_uint_lit(num_bytes, value)

Write a ‘num-bytes’ long integer. The complement of read_uint_lit (A.3.4).

Throws an OutOfRangeError if the value is too large to fit in the requested number of bytes.

write_bitarray(bits, value)

Write the ‘bits’ from the :py;class:bitarray.bitarray ‘value’.

Throws an OutOfRangeError if the value is longer than ‘bits’. The value will be right-hand zero-padded to the required length.

write_bytes(num_bytes, value)

Write the provided bytes or bytearray in a python bytestring.

If the provided byte string is too long an OutOfRangeError will be raised. If it is too short, it will be right-hand zero-padded.

write_uint(value)

Write an unsigned exp-golomb code.

An OutOfRangeError will be raised if a negative value is provided.

write_sint(value)

Signed version of write_uint().

The following utility functions are also provided for converting between offsets given as (bytes, bits) pairs and offsets given in bytes.

to_bit_offset(bytes, bits=7)

Convert from a (bytes, bits) tuple (as used by BitstreamReader.tell() and BitstreamWriter.tell()) into a total number of bits.

from_bit_offset(total_bits)

Convert from a bit offset into a (bytes, bits) tuple (as used by BitstreamReader.tell() and BitstreamWriter.tell()).

Fixeddicts and pseudocode

The vc2_conformance.bitstream.vc2_fixeddicts module contains fixeddict definitions for holding VC-2 bitstream values in a hierarchy which strongly mimics the bitstream structure. These names are re-exported in the vc2_conformance.bitstream module for convenience. See Deserialised VC-2 bitstream data types for a listing.

It also provides the following metadata structures:

vc2_fixeddict_nesting

A lookup {fixeddict_type: [fixeddict_type, ...], ...}.

This lookup enumerates the fixeddicts which may be directly contained by the fixed dictionary types in this module.

This hierarchical information is used by user-facing tools to allow (e.g.) recursive selection of a particular dictionary to display.

vc2_default_values

A lookup {fixeddict_type: {key: default_value, ...}, ...}

For each fixeddict type below, provides a sensible default value for each key. The defaults are generally chosen to produce a minimal, but valid, bitstream.

Where a particular fixeddict entry is a list, the value listed in this lookup should be treated as the default value to use for list entries.

Warning

For default values containing a bitarray.bitarray or any other mutable type, users must take care to copy the default value before mutating it.

The vc2_conformance.bitstream.vc2 module contains a set of SerDes-using (see serdes) functions which follow the pseudo-code in the VC-2 specification as closely as possible. All pseudocode functions are re-exported by the vc2_conformance.bitstream module.

See the table in Deserialised VC-2 bitstream data types which relates these functions to their matching fixeddicts.

In this module, all functions are derived from the pseudocode by:

• Replacing read_* calls with SerDes calls.

• Adding SerDes annotations.

• Removing of decoding functionality (retaining only the code required for bitstream deserialisation).

Consistency with the VC-2 pseudocode is checked by the test suite (see verification).

Autofill

The vc2_conformance.bitstream.vc2_autofill module provides auto-fill routines for automatically computing certain values for the context dictionaries used by the vc2_conformance.bitstream.vc2 serdes functions. These values include the picture number and parse offset fields which can’t default to a simple fixed value.

In the common case, the autofill_and_serialise_stream() function may be used to serialise a complete Stream, with sensible defaults provided for all fields (including picture numbers and next/previous parse offsets).

autofill_and_serialise_stream(file, stream)

Given a Stream dictionary describing a VC-2 stream, serialise that into the supplied file.

Parameters

filefile-like object

A file open for binary writing. The serialised bitstream will be written to this file.

streamStream

The stream to be serialised. Unspecified values will be auto-filled if possible. See Deserialised VC-2 bitstream data types for the default auto-fill values.

Note

Internally, auto-fill values are taken from vc2_default_values_with_auto.

Supported fields containing the special value AUTO will be autofilled with suitably computed values. Specifically:

• Picture numbers will set to incrementing values (starting at 0, or continuing from the value used by the previous picture) by autofill_picture_number().

• The major_version field will be populated by autofill_major_version() and, if appropriate, extended transform parameters fields will be removed.

• Next and previous parse offsets will be calculated automatically by autofill_parse_offsets() and autofill_parse_offsets_finalize().

Autofill value routines

The following functions implement autofill routines for specific bitstream values.

autofill_picture_number(stream, initial_picture_number=0)

Given a Stream, find all picture_number fields which are absent or contain the AUTO sentinel and automatically fill them with consecutive picture numbers. Numbering is restarted for each sequence.

autofill_major_version(stream)

Given a Stream, find all major_version fields which are set to the AUTO sentinel and automatically set them to the appropriate version number for the features used by this stream.

As a side effect, this function will automatically remove the ExtendedTransformParameters field whenever it appears in TransformParameters if the major_version evaluates to less than 3. This change will only be made when major_version was set to AUTO in a proceeding sequence header, if the field was explicitly set to a particular value, no changes will be made to any transform parameters dicts which follow.

autofill_parse_offsets(stream)

Given a Stream, find and fill in all next_parse_offset and previous_parse_offset fields which are absent or contain the AUTO sentinel.

In many (but not all) cases computing these field values is most straight-forwardly done post serialisation. In these cases, fields in the stream will be autofilled with ‘0’. These fields should then subsequently be ammended by autofill_parse_offsets_finalize().

autofill_parse_offsets_finalize(bitstream_writer, stream, next_parse_offsets_to_autofill, previous_parse_offsets_to_autofill)

Finalize the autofillling of next and previous parse offsets by directly modifying the serialised bitstream.

Parameters

bitstream_writerBitstreamWriter

A BitstreamWriter set up to write to the already-serialised bitstream.

streamStream

The context dictionary used to serialies the bitstream. Since computed values added to these dictionaries by the serialisation process, it may be necessary to use the dictionary provided by vc2_conformance.bitstream.Serialiser.context, rather than the one passed into the Serialiser. This is because the Serialiser may have replaced some dictionaries during serialisation.

next_parse_offsets_to_autofill, previous_parse_offsets_to_autofill

The arrays of parse info indices whose next and previous parse offsets remain to be auto-filled.

Autofill value dictionary

vc2_default_values_with_auto

Like vc2_conformance.bitstreams.vc2_default_values but with AUTO set as the default value for all fields which support it.

AUTO

A constant which may be placed in a vc2_fixeddicts fixed dictionary field to indicate that the various autofill_* functions in this module should automatically compute a value for that field.

Metadata

The vc2_conformance.bitstream.metadata module contains metadata about the relationship between VC-2 pseudocode functions and deserialised fixeddict structures. These are used by the vc2-bitstream-viewer command to produce richer output and during documentation generation.

pseudocode_function_to_fixeddicts

For the subset of pseudocode functions in the VC-2 spec dedicated to serialisation/deserialisation, gives the corresponding fixeddict type in vc2_conformance.bitstream.vc2_fixeddicts.

A dictionary of the shape {function_name: [type, ...], ...}.

pseudocode_function_to_fixeddicts_recursive

Like pseudocode_function_to_fixeddicts but each entry also recursively includes the fixeddict types of all contained entries.

fixeddict_to_pseudocode_function

Provides a mapping from vc2_conformance.bitstream.vc2_fixeddicts types to the name of the corresponding pseudocode function which may be used to serialise/deserialise from/to it.