Skip to content

Commit

Permalink
Add orientation metadata to VideoDecoderConfig, update algorithms
Browse files Browse the repository at this point in the history
As proposed on the issue and discussed in WG, this does the following:
- Adds rotation/flip fields to VideoDecoderConfig.
- Updates the "Create a VideoFrame" algorithm used by the video
and image decoders to accept these fields.
- Updates the video and image decoding algorithms to provide
these fields during decoding.

Bug: #351
  • Loading branch information
dalecurtis committed Jan 28, 2025
1 parent b9a8e34 commit 80f011b
Showing 1 changed file with 31 additions and 12 deletions.
43 changes: 31 additions & 12 deletions index.src.html
Original file line number Diff line number Diff line change
Expand Up @@ -922,10 +922,13 @@
5. If {{VideoDecoderConfig/colorSpace}} [=map/exists=] in the
{{VideoDecoder/[[active decoder config]]}}, assign its value to
|colorSpace|.
6. Let |frame| be the result of running the [=Create a VideoFrame=]
6. Assign the values of {{VideoDecoderConfig/rotation}} and
{{VideoDecoderConfig/flip}} to |rotation| and |flip| respectively.
7. Let |frame| be the result of running the [=Create a VideoFrame=]
algorithm with |output|, |timestamp|, |duration|,
|displayAspectWidth|, |displayAspectHeight|, and |colorSpace|.
7. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
|displayAspectWidth|, |displayAspectHeight|, |colorSpace|,
|rotation|, and |flip|.
8. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
</dd>
<dt><dfn>Reset VideoDecoder</dfn> (with |exception|)</dt>
<dd>
Expand Down Expand Up @@ -1970,6 +1973,8 @@
VideoColorSpaceInit colorSpace;
HardwareAcceleration hardwareAcceleration = "no-preference";
boolean optimizeForLatency;
double rotation = 0;
boolean flip = false;
};
</xmp>

Expand Down Expand Up @@ -2059,6 +2064,14 @@
bitstreams require a minimum number of inputs before any output can be
produced.
</dd>
<dt><dfn dict-member for=VideoDecoderConfig>rotation</dfn></dt>
<dd>
Sets the {{VideoFrame/rotation}} attribute on decoded frames.
</dd>
<dt><dfn dict-member for=VideoDecoderConfig>flip</dfn></dt>
<dd>
Sets the {{VideoFrame/flip}} attribute on decoded frames.
</dd>
</dl>


Expand Down Expand Up @@ -3910,7 +3923,7 @@
with {{VideoFrame/[[metadata]]}}.

### Algorithms ###{#videoframe-algorithms}
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|)
: <dfn>Create a VideoFrame</dfn> (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, |colorSpace|, |rotation|, and |flip|)
:: 1. Let |frame| be a new {{VideoFrame}}, constructed as follows:
1. Assign `false` to {{platform object/[[Detached]]}}.
2. Let |resource| be the [=media resource=] described by |output|.
Expand Down Expand Up @@ -3940,6 +3953,8 @@
11. Assign {{VideoFrame/[[color space]]}} with the result of running the
[=VideoFrame/Pick Color Space=] algorithm, with |colorSpace| and
{{VideoFrame/[[format]]}}.
12. Assign {{VideoFrame/rotation}} and {{VideoFrame/flip}} to |rotation|
and |flip| respectively.
2. Return |frame|.


Expand Down Expand Up @@ -5742,9 +5757,11 @@
3. If a |timestamp| can otherwise be trivially generated from
metadata without further decoding, assign that to |timestamp|.
4. Otherwise, assign `0` to |timestamp|.
4. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
|duration|.
4. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
describe it as |rotation| and |flip|.
5. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|,
|duration|, |rotation|, and |flip|.
12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
|decodeResult|.

Expand Down Expand Up @@ -5808,11 +5825,13 @@
3. If a |timestamp| can otherwise be trivially generated from
metadata without further decoding, assign that to |timestamp|.
4. Otherwise, assign `0` to |timestamp|.
17. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
|duration|.
18. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
19. Resolve |promise| with |decodeResult|.
17. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata
describe it as |rotation| and |flip|.
18. Assign {{ImageDecodeResult/image}} with the result of running the
[=Create a VideoFrame=] algorithm with |output|, |timestamp|,
|duration|, |rotation|, and |flip|.
19. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
20. Resolve |promise| with |decodeResult|.

: <dfn for=ImageDecoder>Resolve Decode</dfn> (with |promise| and |result|)
:: 1. [=Queue a task=] to perform these steps:
Expand Down

0 comments on commit 80f011b

Please sign in to comment.