From 80f011ba3aaec0e490a0fd77d1f281ff5af27819 Mon Sep 17 00:00:00 2001 From: Dale Curtis Date: Mon, 27 Jan 2025 15:54:32 -0800 Subject: [PATCH 1/2] Add orientation metadata to VideoDecoderConfig, update algorithms As proposed on the issue and discussed in WG, this does the following: - Adds rotation/flip fields to VideoDecoderConfig. - Updates the "Create a VideoFrame" algorithm used by the video and image decoders to accept these fields. - Updates the video and image decoding algorithms to provide these fields during decoding. Bug: https://github.com/w3c/webcodecs/issues/351 --- index.src.html | 43 +++++++++++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/index.src.html b/index.src.html index 5567601a..970b261d 100644 --- a/index.src.html +++ b/index.src.html @@ -922,10 +922,13 @@ 5. If {{VideoDecoderConfig/colorSpace}} [=map/exists=] in the {{VideoDecoder/[[active decoder config]]}}, assign its value to |colorSpace|. - 6. Let |frame| be the result of running the [=Create a VideoFrame=] + 6. Assign the values of {{VideoDecoderConfig/rotation}} and + {{VideoDecoderConfig/flip}} to |rotation| and |flip| respectively. + 7. Let |frame| be the result of running the [=Create a VideoFrame=] algorithm with |output|, |timestamp|, |duration|, - |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|. - 7. Invoke {{VideoDecoder/[[output callback]]}} with |frame|. + |displayAspectWidth|, |displayAspectHeight|, |colorSpace|, + |rotation|, and |flip|. + 8. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
Reset VideoDecoder (with |exception|)
@@ -1970,6 +1973,8 @@ VideoColorSpaceInit colorSpace; HardwareAcceleration hardwareAcceleration = "no-preference"; boolean optimizeForLatency; + double rotation = 0; + boolean flip = false; }; @@ -2059,6 +2064,14 @@ bitstreams require a minimum number of inputs before any output can be produced.
+
rotation
+
+ Sets the {{VideoFrame/rotation}} attribute on decoded frames. +
+
flip
+
+ Sets the {{VideoFrame/flip}} attribute on decoded frames. +
@@ -3910,7 +3923,7 @@ with {{VideoFrame/[[metadata]]}}. ### Algorithms ###{#videoframe-algorithms} -: Create a VideoFrame (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, and |colorSpace|) +: Create a VideoFrame (with |output|, |timestamp|, |duration|, |displayAspectWidth|, |displayAspectHeight|, |colorSpace|, |rotation|, and |flip|) :: 1. Let |frame| be a new {{VideoFrame}}, constructed as follows: 1. Assign `false` to {{platform object/[[Detached]]}}. 2. Let |resource| be the [=media resource=] described by |output|. @@ -3940,6 +3953,8 @@ 11. Assign {{VideoFrame/[[color space]]}} with the result of running the [=VideoFrame/Pick Color Space=] algorithm, with |colorSpace| and {{VideoFrame/[[format]]}}. + 12. Assign {{VideoFrame/rotation}} and {{VideoFrame/flip}} to |rotation| + and |flip| respectively. 2. Return |frame|. @@ -5742,9 +5757,11 @@ 3. If a |timestamp| can otherwise be trivially generated from metadata without further decoding, assign that to |timestamp|. 4. Otherwise, assign `0` to |timestamp|. - 4. Assign {{ImageDecodeResult/image}} with the result of running the - [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and - |duration|. + 4. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata + describe it as |rotation| and |flip|. + 5. Assign {{ImageDecodeResult/image}} with the result of running the + [=Create a VideoFrame=] algorithm with |output|, |timestamp|, + |duration|, |rotation|, and |flip|. 12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and |decodeResult|. @@ -5808,11 +5825,13 @@ 3. If a |timestamp| can otherwise be trivially generated from metadata without further decoding, assign that to |timestamp|. 4. Otherwise, assign `0` to |timestamp|. - 17. Assign {{ImageDecodeResult/image}} with the result of running the - [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and - |duration|. - 18. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 19. Resolve |promise| with |decodeResult|. + 17. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata + describe it as |rotation| and |flip|. + 18. Assign {{ImageDecodeResult/image}} with the result of running the + [=Create a VideoFrame=] algorithm with |output|, |timestamp|, + |duration|, |rotation|, and |flip|. + 19. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 20. Resolve |promise| with |decodeResult|. : Resolve Decode (with |promise| and |result|) :: 1. [=Queue a task=] to perform these steps: From 0184fb511da6813528716a4b6400d3ad46697111 Mon Sep 17 00:00:00 2001 From: Dale Curtis Date: Tue, 28 Jan 2025 16:28:14 -0800 Subject: [PATCH 2/2] Handle case with no rotation/flip for ImageDecoder. --- index.src.html | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/index.src.html b/index.src.html index 970b261d..a30aca64 100644 --- a/index.src.html +++ b/index.src.html @@ -5758,7 +5758,8 @@ metadata without further decoding, assign that to |timestamp|. 4. Otherwise, assign `0` to |timestamp|. 4. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata - describe it as |rotation| and |flip|. + describe it as |rotation| and |flip|, otherwise set |rotation| to 0 + and |flip| to false. 5. Assign {{ImageDecodeResult/image}} with the result of running the [=Create a VideoFrame=] algorithm with |output|, |timestamp|, |duration|, |rotation|, and |flip|. @@ -5826,7 +5827,8 @@ metadata without further decoding, assign that to |timestamp|. 4. Otherwise, assign `0` to |timestamp|. 17. If {{ImageDecoder/[[encoded data]]}} contains orientation metadata - describe it as |rotation| and |flip|. + describe it as |rotation| and |flip|, otherwise set |rotation| to 0 + and |flip| to false. 18. Assign {{ImageDecodeResult/image}} with the result of running the [=Create a VideoFrame=] algorithm with |output|, |timestamp|, |duration|, |rotation|, and |flip|.