vc2_conformance.test_cases: VC-2 codec test case generation

The vc2_conformance.test_cases module contains routines for generating test cases for VC-2 encoder and decoder implementations.

A description of each test case generator can be found in Encoder test cases and Decoder Test Cases in the user guide.

Test case generators

Test case generators are generator functions which take a CodecFeatures dictionary as their only argument and produce picture sequences (for encoders) or bitstreams (for decoders). Test case generators may take one of the following forms:

  • Function which returns a single test case

  • Function which returns a single test case or None (indicating no test case was produced)

  • Generator function which yields multiple test cases.

A test case can be one of:

Test case generators may prefer to output TestCase objects when multiple test cases are produced so that each testcase can be given its own ‘subcase’ name. In addition, test cases may be accompanied by a freeform JSON serialisable metadata object when TestCase objects are produced.

class TestCase(value, subcase_name=None, case_name=None, metadata=None)

A test case, produced by a test case generator function.

Parameters
valueEncoderTestSequence or Stream

A value containing the test case itself. An EncoderTestSequence for encoder test cases or Stream for decoder test cases.

subcase_namestr or None

An identifier for the sub-case if this test case has several sub-cases.

case_namestr or None

The name of the test case. If None, the case_name will be populated automatically with the name of the test case generator function which returned it.

metadataobject or None

Optional JSON-serialisable metadata associated with this test case. The meaning and formatting of of this metadata is left to the individual test case to define.

property name

The complete name of this test case. Constructed from the case_name and subcase_name attributes.

A string of the form "case_name" or "case_name[subcase_name]".

property case_name
property subcase_name
property value
property metadata

All test case generator functions are run via normalise_test_case_generator() which normalises the function into the form of a generator which yields TestCase objects. This also populates the case_name field of the generated TestCase automatically with the test case generator’s function name.

normalise_test_case_generator(f, *args, **kwargs)

Call a test case generator, f, and, regardless of its native output, produces a generator of TestCase objects.

If the function returns or yields TestCase objects, their TestCase.case_name attributes will be populated with the function name, if not already defined. If the function returns or generates other values, these will be wrapped in TestCase objects automatically. If the function returns or generates None, no test case will be emitted.

Encoder test case generators

Encoder test case generators are located in vc2_conformance.test_cases.encoder must be decorated with the vc2_conformance.test_cases.encoder_test_case_generator decorator, take a CodecFeatures and produce EncoderTestSequence objects.

encoder_test_case_generator

Decorator to use to register all encoder test case generators.

class EncoderTestSequence(pictures, video_parameters, picture_coding_mode)

A sequence of pictures to be encoded by a VC-2 encoder under test.

Parameters
pictures[{“Y”: [[s, …], …], “C1”: …, “C2”: …, “pic_num”: int}, …]

A list of dictionaries containing raw picture data in 2D arrays for each picture component.

video_parametersVideoParameters

The video parameters associated with the test sequence.

picture_coding_modePictureCodingModes

The picture coding mode associated with the test sequence.

Decoder test case generators

Decoder test case generators are located in vc2_conformance.test_cases.decoder must be decorated with the vc2_conformance.test_cases.decoder_test_case_generator decorator, take a CodecFeatures and produce vc2_conformance.bitstream.Stream dictionaries which can be serialised using autofill_and_serialise_stream().

decoder_test_case_generator

Decorator to use to register all decoder test case generators.

Test case generator registries

All test case generators are registered (by the encoder_test_case_generator and decoder_test_case_generator decorators) with one of two Registry singletons:

ENCODER_TEST_CASE_GENERATOR_REGISTRY

Registry singleton with which all VC-2 encoder test cases are registered.

DECODER_TEST_CASE_GENERATOR_REGISTRY

Registry singleton with which all VC-2 decoder test cases are registered.

The Registry class implements a registry of test case generators which the vc2-test-case-generator script uses to generate a complete set of test cases.

class Registry

A registry of test case generating functions.

register_test_case_generator(f)

Register a test case generator function with this registry.

Returns the (unmodified) function allowing this method to be used as a decorator.

generate_test_cases(*args, **kwargs)

Run every test case generator registered with this registry, passing each generator the supplied arguments. Generates all TestCase objects.

iter_independent_generators(*args, **kwargs)

Produce a series of generator functions which may be called in parallel. Each returned zero-argument function should be called and will generate a series of TestCase objects.

iter_registered_functions()

Iterates over the raw functions registered with this registry.

Only intended for use during documentation generation.