Clarify the color info in the codecs string #142

wantehchang · 2022-04-04T16:57:06Z

cconcolato · 2022-05-02T17:37:42Z

The changes in the first part (#126) implement what was suggested in the issue. We realize this is a normative change with potential backwards compatibility issues. We need to check with implementations.

For the codecs parameter changes (#129), we would prefer:

bullet point lists and indentation for readability
having the range conditions separate as they do not dependent on the color present flag
This is also a normative change but fixing a case where there was conflicting information between the codecs parameter and the bitstream. We need careful review.

In both cases, we should document before and after behavior.

wantehchang · 2022-06-13T16:57:35Z

Here is a screen shot of the first change in this pull request:

Here is a screen shot of the second change in this pull request. My change ends with the paragraph "The videoFullRangeFlag is represented by a single digit." The screen shot includes a few paragraphs below and the table of default values as context.

cconcolato · 2022-06-13T17:17:20Z

We should update the PR again to cover the case of monochrome and chromaSubsampling. They cannot be present if the following fields are omitted (and vice versa).

cconcolato · 2022-06-13T17:19:01Z

The codecs parameter is also allowed to omit the trailing values if the colr box or the sequence header OBU contains the default values.

cconcolato · 2022-06-13T17:22:20Z

We also need to document the before and after cases, showing possible backwards compatibility changes.

wantehchang · 2022-10-11T00:00:59Z

The new version of the pull request has addressed all the review comments except the following:

The value of the videoFullRangeFlag is not specified separately. This results in a little duplicate text but I think overall this is clearer.
I didn't document the before and after cases, showing possible backwards compatibility changes.

podborski · 2022-10-11T01:43:30Z

Question, should videoFullRangeFlag be color_range here as well? (similar fix as in HDR10+ pull request?) What about other terms?

wantehchang · 2022-11-14T17:59:07Z

Screenshot of the top half of Section 5:

wantehchang · 2022-11-14T17:59:41Z

Screenshot of the bottom half of Section 5:

cconcolato · 2022-11-14T18:37:58Z

We should split the PR into 2:

one clarifying the colr box and Config OBU part
one clarifying the codecs string

cconcolato · 2022-11-14T18:38:30Z

For the colr, I suggest we clarify along the lines of this document.

The summary for colr is that:

it may be absent, it is not recommended
when it is present, its values are authoritative and may match the SH but may override them, but in the end the values in colr may still be 2/2/2 for cases like DV.

cconcolato · 2022-11-14T18:52:36Z

Regarding the codecs string part, the added sentence seems wrong:

If any of the colorPrimaries, transferCharacteristics, and matrixCoefficients parameter values equals 02 (unspecified), it SHALL be set to the default value below .

It does not enable 2/2/2 for DV for example. Also the "it" is not clear. Simply deleting the sentence is good, or moving it into the "if colr box is absent.
We could add a note that says: if you explicitly want 2/2/2 in the codecs string, you should use a colr box with these 2/2/2 values.

wantehchang · 2023-03-17T23:01:33Z

I wrote the pull request #167 to clarify the colr box.

wantehchang · 2023-05-15T16:48:22Z

I updated this pull request to the current main branch and edited the subject line and commit message accordingly. This pull request now addresses #129 only.

cconcolato · 2023-07-10T17:32:38Z

Some notes on the PR:

we note that there are colr boxes with other types than nclx (e.g. nclc, or prof). The PR considers that anything else that nclx requires going into the bitstream. Is that what we want? Probably.
There are different tools processing codecs strings: MPD generator, players, ... We wonder how complex the generation of the codecs string should be for MPD generator (assuming more work can be done by encoder/packager, when creating the MP4 file from the OBU stream). The alternative would be to never read the sequence header? How many files would this invalidate?
Maybe we should add a note that if you use a color space that is not registered in CICP, if you don't explicitly code 2/2/2 either in the sequence header (with color description present flag = 1) or in the colr box, you will get a codecs string that declares BT 709. Writers should act accordingly.

podborski · 2023-09-18T16:52:44Z

Maybe one more thing to consider. The HLS spec. also defines a VIDEO-RANGE as

VIDEO-RANGE definition

VIDEO-RANGE

  The value is an enumerated-string; valid strings are SDR, HLG and
  PQ.

  The value MUST be SDR if the video in the Variant Stream is
  encoded using one of the following reference opto-electronic
  transfer characteristic functions specified by the
  TransferCharacteristics code point: [[CICP](https://datatracker.ietf.org/doc/html/draft-pantos-hls-rfc8216bis#ref-CICP)] 1, 6, 13, 14, 15.  Note
  that different TransferCharacteristics code points can use the
  same transfer function.

 The value MUST be HLG if the video in the Variant Stream is
  encoded using a reference opto-electronic transfer characteristic
  function specified by the TransferCharacteristics code point 18,
  or consists of such video mixed with video qualifying as SDR (see
  above).

  The value MUST be PQ if the video in the Variant Stream is encoded
  using a reference opto-electronic transfer characteristic function
  specified by the TransferCharacteristics code point 16, or
  consists of such video mixed with video qualifying as SDR or HLG
  (see above).

  This attribute is OPTIONAL.  Its absence implies a value of SDR.
  Clients that do not recognize the attribute value SHOULD NOT
  select the Variant Stream.

I think it would be useful if we at least add a note, mentioning that care needs to be taken as the VIDEO-RANGE attribute in HLS groups certain transfer characteristics CICP values into 3 buckets (SDR,HLG,PQ) and codecs string is signaling similar information.

cconcolato · 2023-09-18T17:29:03Z

We could add a note that says:
NOTE: when the colr box is present, it simplifies the task of the codecs parameter generator, and is encouraged.

wantehchang · 2023-09-21T23:12:12Z

I added the NOTE that Cyril suggested. I also move the common text about color_range to a separate bullet point.

wantehchang · 2023-09-22T01:28:10Z

Cyril,

I have some questions about your comment on Jul 10.

we note that there are colr boxes with other types than nclx (e.g. nclc, or prof).

I assume "nclc" is a typo for "'rICC".

The PR considers that anything else that nclx requires going into the bitstream. Is that what we want? Probably.

Could you clarify what you meant by "anything else that nclx requires going into the bitstream"? I wonder if you meant to say "anything other than nclx requires going into the bitstream".

By "going into the bitstream", you meant going into the Sequence Header OBU, right?

Maybe we should add a note that if you use a color space that is not registered in CICP, if you don't explicitly code 2/2/2 either in the sequence header (with color description present flag = 1) or in the colr box, you will get a codecs string that declares BT 709. Writers should act accordingly.

Would you still like me to add this note?

Thanks.

Fix AOMediaCodec#129.

This enables 2/2/2 for DV.

If color_description_present_flag is set to 0 in the Sequence Header OBU, the colorPrimaries, transferCharacteristics, matrixCoefficients parameter values SHOULD be set to the default values 01.01.01. This change maintains backward compatibility with v1.2.0, which says (after paraphrasing): if color_description_present_flag is set to 0, the colorPrimaries, transferCharacteristics, matrixCoefficients, and videoFullRangeFlag parameter values SHOULD not be set, defaulting to the values 01.01.01.0. NOTE: The color_range flag is always present in the Sequence Header OBU, so the current draft specifies that the videoFullRangeFlag parameter value SHALL be set to the color_range flag in the Sequence Header OBU.

to be consistent with the prevalent style in this section.

wantehchang · 2023-09-22T17:43:28Z

Dimitri wrote:

Maybe one more thing to consider. The HLS spec. also defines a VIDEO-RANGE [...snipped...]

I think it would be useful if we at least add a note, mentioning that care needs to be taken as the VIDEO-RANGE attribute in HLS groups certain transfer characteristics CICP values into 3 buckets (SDR,HLG,PQ) and codecs string is signaling similar information.

Should I add the note that Dimitri suggested?

The HLS spec that Dimitri mentioned is still an Internet Draft. It seems premature to mention it.

cconcolato · 2023-09-25T18:23:38Z

I assume "nclc" is a typo for "'rICC".

No. This was an example. nclc exists, typically in QuickTime files.

cconcolato · 2023-09-25T18:24:23Z

Could you clarify what you meant by "anything else that nclx requires going into the bitstream"? I wonder if you meant to say "anything other than nclx requires going into the bitstream".

Yes, that's what I meant.

By "going into the bitstream", you meant going into the Sequence Header OBU, right?

Yes.

cconcolato · 2023-09-25T18:25:32Z

Would you still like me to add this note?

Yes, that couldn't hurt.

cconcolato · 2023-09-25T18:26:25Z

Should I add the note that Dimitri suggested?

No. We said that Dimitri will open a separate issue to discuss HLS. Let's not overload this PR too much.

wantehchang · 2023-09-29T23:19:51Z

OK, I have addressed all the comments on this pull request.

index.bs

Co-authored-by: Cyril Concolato <[email protected]>

cconcolato · 2023-10-02T18:29:22Z

Unless objections in the coming days I will merge this PR. Thank you @wantehchang

rafaelcaricio · 2023-11-05T19:01:30Z

index.bs

+
+The videoFullRangeFlag is represented by a single digit.
+
+<assert>If the codecs parameter string ends with ".0.110.01.01.01.0" (containing all the default values below), that trailing part of the string SHOULD be omitted</assert>.


Maybe this could be changed from SHOULD to MAY? I think it would be an implementers choice to include it or not.

Thanks for the comment. It's a SHOULD rather than a MAY because it is inefficient to include all the default values. But in the end, it's an implementer's choice. The SHOULD just gives more indication as to what needs to be done than a MAY. Unless we have a use case that needs to include the default values, I think we should keep the SHOULD.

cconcolato · 2023-11-06T00:42:10Z

Yes
No

What are you commenting on?

…tring-note SHA: 014d8ea Reason: push, by cconcolato Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from e7dc5d5 to 09c653f Compare April 4, 2022 16:57

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from 09c653f to 0d05719 Compare June 13, 2022 16:50

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from 0d05719 to 5e7c696 Compare August 8, 2022 16:49

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch 3 times, most recently from 2eb4e0f to 3d430f4 Compare October 10, 2022 23:53

wantehchang requested review from cconcolato and podborski October 10, 2022 23:58

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from 3d430f4 to 41a67a8 Compare May 15, 2023 16:43

wantehchang changed the title ~~Clarify 'colr' box and color info in codecs string~~ Clarify the color info in the codecs string May 15, 2023

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch 2 times, most recently from 141ac84 to 523bc57 Compare June 26, 2023 01:59

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from 523bc57 to 9a066ee Compare August 7, 2023 17:34

wantehchang added 5 commits September 22, 2023 10:28

Clarify the color info in the codecs string

1d463dd

Fix AOMediaCodec#129.

Allow 02 in the CICP part of codecs string

879b5ab

This enables 2/2/2 for DV.

Change "be set to" to "equal"

df3e498

to be consistent with the prevalent style in this section.

Describe color_range separately,encourage colr box

7fc9dea

wantehchang force-pushed the issue-129-codecs-parameter-string-note branch from c23000b to 7fc9dea Compare September 22, 2023 17:29

Add note stressing codecs string default is BT.709

3191010

cconcolato reviewed Oct 2, 2023

View reviewed changes

index.bs Outdated Show resolved Hide resolved

wantehchang and others added 2 commits October 2, 2023 10:23

Rephrase the second NOTE.

e843533

Co-authored-by: Cyril Concolato <[email protected]>

Add an item for CICP (ITU-T H.273) to "biblio"

537ba21

cconcolato approved these changes Oct 2, 2023

View reviewed changes

podborski mentioned this pull request Oct 3, 2023

On HLS VIDEO-RANGE #177

Open

Switch to a stable URL of CICP (ITU-T H.273)

b64c7de

rafaelcaricio reviewed Nov 5, 2023

View reviewed changes

cconcolato merged commit 014d8ea into AOMediaCodec:main Nov 27, 2023


		The videoFullRangeFlag is represented by a single digit.

		<assert>If the codecs parameter string ends with ".0.110.01.01.01.0" (containing all the default values below), that trailing part of the string SHOULD be omitted</assert>.

Clarify the color info in the codecs string #142

Clarify the color info in the codecs string #142

Uh oh!

Conversation

wantehchang commented Apr 4, 2022 • edited by pr-preview bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cconcolato commented May 2, 2022

Uh oh!

wantehchang commented Jun 13, 2022

Uh oh!

cconcolato commented Jun 13, 2022

Uh oh!

cconcolato commented Jun 13, 2022

Uh oh!

cconcolato commented Jun 13, 2022

Uh oh!

wantehchang commented Oct 11, 2022

Uh oh!

podborski commented Oct 11, 2022

Uh oh!

wantehchang commented Nov 14, 2022

Uh oh!

wantehchang commented Nov 14, 2022

Uh oh!

cconcolato commented Nov 14, 2022

Uh oh!

cconcolato commented Nov 14, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cconcolato commented Nov 14, 2022

Uh oh!

wantehchang commented Mar 17, 2023

Uh oh!

wantehchang commented May 15, 2023

Uh oh!

cconcolato commented Jul 10, 2023

Uh oh!

podborski commented Sep 18, 2023

Uh oh!

cconcolato commented Sep 18, 2023

Uh oh!

wantehchang commented Sep 21, 2023

Uh oh!

wantehchang commented Sep 22, 2023

Uh oh!

wantehchang commented Sep 22, 2023

Uh oh!

cconcolato commented Sep 25, 2023

Uh oh!

cconcolato commented Sep 25, 2023

Uh oh!

cconcolato commented Sep 25, 2023

Uh oh!

cconcolato commented Sep 25, 2023

Uh oh!

wantehchang commented Sep 29, 2023

Uh oh!

Uh oh!

cconcolato commented Oct 2, 2023

Uh oh!

rafaelcaricio Nov 5, 2023

Choose a reason for hiding this comment

Uh oh!

cconcolato Nov 6, 2023

Choose a reason for hiding this comment

Uh oh!

cconcolato commented Nov 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wantehchang commented Apr 4, 2022 •

edited by pr-preview bot

Loading

cconcolato commented Nov 14, 2022 •

edited

Loading