vc2_conformance.decoder
: Reference decoder and bitstream validator¶
The vc2_conformance.decoder
module contains the components of a VC-2
decoder and bitstream validator. Along with the basic decoding logic,
additional tests are included to check all conditions imposed on bitstreams by
the VC-2 specification.
Usage¶
The bitstream decoder/validator is exposed to end-users via the vc2-bitstream-validator command line utility.
This module may also be used directly. The following snippet illustrates how a VC-2 bitstream might be decoded and verified using this module:
>>> from vc2_conformance.string_utils import wrap_paragraphs
>>> from vc2_conformance.pseudocode import State
>>> from vc2_conformance.decoder import init_io, parse_stream, ConformanceError
>>> # Create a callback to be called with picture data whenever a picture
>>> # is decoded from the bitstream.
>>> def output_picture_callback(picture, video_parameters, picture_coding_mode):
>>> print("A picture was decoded...")
>>> # Create an initial state object ready to read the bitstream
>>> state = State(_output_picture_callback=output_picture_callback)
>>> f = open("path/to/bitstream.vc2", "rb")
>>> init_io(state, f)
>>> # Decode and validate!
>>> try:
... parse_stream(state)
... print("Bitstream is valid!")
... except ConformanceError as e:
... print("Bitstream is NOT valid:")
... print(wrap_paragraphs(e.explain(), 80))
Bitstream is NOT valid:
An invalid parse code, 0x0A, was provided to a parse info header (10.5.1).
See (Table 10.1) for the list of allowed parse codes.
Perhaps this bitstream conforms to an earlier or later version of the VC-2
standard?
Overview¶
This decoder is based on the pseudocode published in the VC-2 specification and
consequently follows the same structure as the pseudocode with the
parse_stream()
function being used to
decode a complete stream.
The pseudocode is automatically verified for consistency with the VC-2
specification by the conformance software test suite. Verified pseudocode
functions are annotated with the
ref_pseudocode()
decorator. See
verification
(in the tests/
directory) for details on the
automated verification process. All bitstream validation logic, which doesn’t
form part of the specified pseudocode, appears between ## Begin not in spec
and ## End not in spec
comments.
All global state is passed around via the
State
dictionary. This dictionary
is augmented with a number of additional entries not included in the VC-2
specification but which are necessary for a ‘real’ decoder implementation (e.g.
an input file handle) and for validation purposes (e.g. recorded offsets of
previous data units). See vc2_conformance.pseudocode.state.State
for a complete enumeration of these.
Underlying I/O operations are not specified by the VC-2 specification. This
decoder reads streams from file-like objects. See
the vc2_conformance.decoder.io
module for details. As illustrated in
the example above, the init_io()
function
is used to specify the file-like object the stream will be read from.
When conformance errors are detected,
ConformanceError
exceptions are
thrown. These exceptions provide in-depth human readable explanations of
conformance issues along with suggested invocations of the
vc2-bitstream-viewer tool for diagnosing issues. See the
vc2_conformance.decoder.exceptions
module for details. These
exceptions are largely thrown directly by validation code spliced into the
pseudocode routines. Some common checks are factored out into their own
‘assertions’ in the vc2_conformance.decoder.assertions
.
The decoder logic is organised in line with the sections of the VC-2 specification:
vc2_conformance.decoder.io
: (A) Bitstream I/Ovc2_conformance.decoder.stream
: (10) Stream syntaxvc2_conformance.decoder.sequence_header
: (11) Sequence headervc2_conformance.decoder.picture_syntax
: (12) Picture syntaxvc2_conformance.decoder.transform_data_syntax
: (13) Transform data syntaxvc2_conformance.decoder.fragment_syntax
: (14) Fragment syntax
The vc2_conformance.decoder
module only includes pseudocode functions
which define additional behaviour not defined by the spec. Specifically, this
includes performing I/O or additional checks for bitstream validation purposes.
All other pseudocode routines are used ‘verbatim’ and can be found in
vc2_conformance.pseudocode
.
Stream I/O¶
The vc2_conformance.io
module implements I/O functions for reading
bitstreams from file-like objects. The exposed functions implement the
interface specified in annex (A).
Initialisation¶
The init_io()
function must be used to initialise a
State
dictionary so that it is
ready to read a bitstream.
-
init_io
(state, f)¶ (A.2.1) Initialise the I/O-related variables in state.
This function should be called exactly once to initialise the I/O-related parts of the state dictionary to their initial state as specified by (A.2.1):
… a decoder is deemed to maintain a copy of the current byte, state[current_byte], and an index to the next bit (in the byte) to be read, state[next_bit] …
As well as initialising the state[“current_byte”] and state[“next_bit”] fields, this sets the (out-of-spec) state[“_file”] entry to the provided file-like object.
- Parameters
- state
State
The state dictionary to be initialised.
- ffile-like object
The file to read the bitstream from.
- state
Determining stream position¶
The tell()
function (below) is used by verification logic to report
and check offsets of values within a bitstream. For example, it may be used to
check next_parse_offset
fields are correct (see
vc2_conformance.decoder.stream.parse_info()
).
-
tell
(state)¶ Not part of spec; used to log bit offsets in the bitstream.
Return a (byte, bit) tuple giving the offset of the next bit to be read in the stream.
Bitstream recording¶
The VC-2 specification sometimes requires that the coded bitstream
representation of a particular set of repeated fields is consistent within a
bitstream (e.g. see
vc2_conformance.decoder.sequence_header.sequence_header()
). To
facilitate this test, the record_bitstream_start()
and
record_bitstream_finish()
functions may be used to capture the
bitstream bytes read within part of the bitstream.
-
record_bitstream_start
(state)¶ Not part of spec; used for verifying that repeated sequence_headers are byte-for-byte identical (11.1).
This function causes all future bytes read from the bitstream to be logged into state[“_read_bytes”] until
record_bitstream_finish()
is called.Recordings must start byte aligned.
-
record_bitstream_finish
(state)¶ -
- Returns
- bytearray
The bytes read since
record_bitstream_start()
was called. Any unread bits of the final byte will be set to zero.
Conformance exceptions¶
The vc2_conformance.decoder.exceptions
module defines a number of
exceptions derived from ConformanceError
representing different
conformance errors a bitstream may contain. These exceptions provide additional
methods which return detailed human-readable information about the conformance
error.
-
exception
ConformanceError
¶ Base class for all bitstream conformance failure exceptions.
-
explain
()¶ Produce a detailed human readable explanation of the conformance failure.
Should return a string which can be re-linewrapped by
vc2_conformance.string_utils.wrap_paragraphs()
.The first line will be used as a summary when the exception is printed using
str()
.
-
bitstream_viewer_hint
()¶ Return a set of sample command line arguments for the vc2-bitstream-viewer tool which will display the relevant portion of the bitstream.
This string may include the following
str.format()
substitutions which should be filled in before display:{cmd}
The command name of the bitstream viewer (i.e. usuallyvc2-bitstream-viewer
){file}
The filename of the bitstream.{offset}
The bit offset of the next bit in the bitstream to be read.
This returned string should not be line-wrapped but should be de-indented by
textwrap.dedent()
.
-
offending_offset
()¶ If known, return the bit-offset of the offending part of the bitstream. Otherwise return None (and the current offset will be assumed).
-
Sequence composition restrictions¶
Various restrictions may be imposed on choice and order of data unit types in a sequence. For example, all sequences must start with a sequence header and end with an end of sequence. Some levels impose additional restrictions such as prohibiting the mixing of fragments and pictures or requiring sequence headers to be interleaved between every picture.
Rather than using ad-hoc logic to enforce these restrictions, regular
expressions are used to check the pattern of data unit types. See the
vc2_conformance.symbol_re
module for details on the regular
expression matching system.
Level constraints¶
VC-2’s levels impose additional constraints on bitstreams, for example
restricting some fields to particular ranges of values. Rather than including
ad-hoc validation logic for each level, a ‘constraints table’ is used.
Bitstream values which may be constrained are checked using
vc2_conformance.decoder.assertions.assert_level_constraint()
.
See the vc2_conformance.constraint_table
module for an introduction
to constraint tables. See the vc2_conformance.level_constraints
module for level-related constraint data, including documentation on the
entries included in the levels constraints table.