Introduces Posterization preprocessing layer. #136

sebastian-sz · 2022-02-15T18:40:35Z

Posterization is one of the operations used by AutoAugment and RandAugment.

keras_cv/layers/preprocessing/posterization.py

keras_cv/layers/preprocessing/posterization_test.py

bhack · 2022-02-16T23:40:11Z

keras_cv/layers/preprocessing/posterization_test.py

+
+        self.assertEqual(output.shape, dummy_input.shape)
+
+    def test_output_dtype_unchanged(self):


Probably the same also on this one.

LukeWood · 2022-02-17T22:28:52Z

keras_cv/layers/preprocessing/posterization.py

+            [batch, height, width, channels] or [height, width, channels].
+    """
+
+    def __init__(self, bits: int):


Hey there - please include value_range as an argument and scale images up to [0, 255] according to value_range, then scale back down to value_range before returning. Thank.

LukeWood · 2022-02-17T22:29:30Z

Thank you for the PR. Left a starting comment, will do a thorough review. Please be aware we are doing some major design work around KPLs in KerasCV and the API may shift as a result. Apologies if this delays reviews.

sebastian-sz · 2022-02-18T06:43:48Z

Thank you for the PR. Left a starting comment, will do a thorough review. Please be aware we are doing some major design work around KPLs in KerasCV and the API may shift as a result. Apologies if this delays reviews.

Hi, no worries. Let me know if you're done with refactoring and design changes, I will update this PR accordingly.

sebastian-sz · 2022-03-07T19:05:40Z

@LukeWood I wanted to update the layer to subclass BaseImageAugmentationLayer, but it's not available in TF 2.8. Should I wait for 2.9?

bhack · 2022-03-07T19:18:31Z

@LukeWood I wanted to update the layer to subclass BaseImageAugmentationLayer, but it's not available in TF 2.8. Should I wait for 2.9?

It seems that we rely on Tensorflow stable in this repo but in Keras repo on master we track TF Nightly:
https://github.com/keras-team/keras-cv/blob/master/setup.py#L27

LukeWood · 2022-03-07T20:03:50Z

@LukeWood I wanted to update the layer to subclass BaseImageAugmentationLayer, but it's not available in TF 2.8. Should I wait for 2.9?

It seems that we rely on Tensorflow stable in this repo but in Keras repo on master we track TF Nightly: https://github.com/keras-team/keras-cv/blob/master/setup.py#L27

I'll update the repo to rely on nightly until 2.9. We're in pre-release so this is fine for now.

LukeWood · 2022-03-10T19:05:48Z

BaseImageAugmentationLayer

Yes, sorry for the delayed response.

sebastian-sz · 2022-03-18T13:09:00Z

@LukeWood I'm starting to use BaseImageAugmentationLayer. I noticed it force-casts all inputs to tf.float32 - is this intended?

import tensorflow as tf
from tensorflow.keras.__internal__.layers import BaseImageAugmentationLayer


class DummyLayer(BaseImageAugmentationLayer):
    def augment_image(self, image, transformation=None):
        return image 

layer = DummyLayer()
image = tf.random.uniform((1, 224, 224, 3),dtype=tf.int32)
output = layer(image)
print(output.dtype)  # tf.float32, but input was tf.int32

LukeWood · 2022-03-18T17:35:44Z

The layer will cast your inputs to self.compute_dtype. This is pretty standard for keras.

LukeWood · 2022-03-18T17:38:45Z

Basically @sebastian-sz to ensure consistency across dtypes we will just always assume that the inputs are float, and make no assumptions about their value range.

LukeWood · 2022-03-21T19:56:37Z

Hey @sebastian-sz a sample usage of BaseImageAugmentationLayer is available in #180.

Feel free to base yours on this. Upload when a copy is ready and I will review :)

sebastian-sz · 2022-03-22T14:02:20Z

Thanks @LukeWood I'm almost done with updating the code but I'm still unhappy with how the tests look.

This is in progress and will probably be done by the end of the week (probably sooner).

LukeWood · 2022-03-22T20:08:19Z

awesome - yeah feel free to upload as you go. I may begin lending a hand in some of these components as the list of components required for RandAugment shrinks.

sebastian-sz · 2022-03-23T06:35:42Z

@LukeWood I updated the PR. Two questions:

What is the tf.function policy? Should the augment_image or any other function be decorated?
Is it ok to override _batch_augment as well if the implementation is already vectorized and does not need to rely on vectorized_map / map_fn ? See code for more details, if not I will remove this part.

LukeWood · 2022-03-23T18:23:35Z

@LukeWood I updated the PR. Two questions:

What is the tf.function policy? Should the augment_image or any other function be decorated?

Is it ok to override _batch_augment as well if the implementation is already vectorized and does not need to rely on vectorized_map / map_fn ? See code for more details, if not I will remove this part.

1.) no need for tf.function
2.) I think it is actually fine. I will check with @qlzh727 too.

LukeWood · 2022-03-23T18:19:01Z

keras_cv/layers/preprocessing/posterization.py

+        return transform_value_range(
+            images=image,
+            original_range=[0, 255],
+            target_range=self._value_range,


lets cast back to compute_dtype before transform_value_range. That way we are certain it is the correct dtype. transform_value_range can be skipped if value_range is [0, 255]

LukeWood · 2022-03-23T18:25:11Z

@LukeWood I updated the PR. Two questions:

What is the tf.function policy? Should the augment_image or any other function be decorated?

Is it ok to override _batch_augment as well if the implementation is already vectorized and does not need to rely on vectorized_map / map_fn ? See code for more details, if not I will remove this part.

1.) no need for tf.function 2.) I think it is actually fine. I will check with @qlzh727 too.

Discussed with Scott - it is fine for now. Thanks! Good idea.

bhack · 2022-03-23T18:32:55Z

1.) no need for tf.function

The model logic is wrapped by default with a tf.function until you will go to explicitly enforce the eager mode like in model compilation

jit_compile could be different instead as we could need to limit the performance region to a specific function. See more at #165

LukeWood · 2022-03-23T18:43:07Z

Thanks a ton @sebastian-sz - great contribution! Looks good to go!

* Added Posterization preprocessing layer. * Changed Posterization's AssertionError to ValueError. * Refactored Posterization to use BaseImageAugmentationLayer. * Copyeditst Co-authored-by: Luke Wood <lukewoodcs@gmail.com>

bhack reviewed Feb 16, 2022

View reviewed changes

keras_cv/layers/preprocessing/posterization.py Outdated Show resolved Hide resolved

bhack reviewed Feb 16, 2022

View reviewed changes

keras_cv/layers/preprocessing/posterization_test.py Outdated Show resolved Hide resolved

bhack reviewed Feb 16, 2022

View reviewed changes

keras_cv/layers/preprocessing/posterization_test.py Outdated Show resolved Hide resolved

bhack reviewed Feb 16, 2022

View reviewed changes

LukeWood reviewed Feb 17, 2022

View reviewed changes

sebastian-sz added 3 commits March 23, 2022 07:27

Added Posterization preprocessing layer.

641bed8

Changed Posterization's AssertionError to ValueError.

d618c83

Refactored Posterization to use BaseImageAugmentationLayer.

e598294

sebastian-sz force-pushed the feature-84/introduce-posterization-layer branch from 91b83ce to e598294 Compare March 23, 2022 06:35

LukeWood suggested changes Mar 23, 2022

View reviewed changes

Copyeditst

831f68a

LukeWood merged commit 1f7bde4 into keras-team:master Mar 23, 2022

sebastian-sz deleted the feature-84/introduce-posterization-layer branch March 30, 2022 08:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduces Posterization preprocessing layer. #136

Introduces Posterization preprocessing layer. #136

sebastian-sz commented Feb 15, 2022

bhack Feb 16, 2022

LukeWood Feb 17, 2022

LukeWood commented Feb 17, 2022

sebastian-sz commented Feb 18, 2022

sebastian-sz commented Mar 7, 2022

bhack commented Mar 7, 2022

LukeWood commented Mar 7, 2022

LukeWood commented Mar 10, 2022

sebastian-sz commented Mar 18, 2022

LukeWood commented Mar 18, 2022

LukeWood commented Mar 18, 2022

LukeWood commented Mar 21, 2022

sebastian-sz commented Mar 22, 2022

LukeWood commented Mar 22, 2022

sebastian-sz commented Mar 23, 2022

LukeWood commented Mar 23, 2022

LukeWood Mar 23, 2022

LukeWood commented Mar 23, 2022

bhack commented Mar 23, 2022

LukeWood commented Mar 23, 2022


		self.assertEqual(output.shape, dummy_input.shape)

		def test_output_dtype_unchanged(self):

Introduces Posterization preprocessing layer. #136

Introduces Posterization preprocessing layer. #136

Conversation

sebastian-sz commented Feb 15, 2022

bhack Feb 16, 2022

Choose a reason for hiding this comment

LukeWood Feb 17, 2022

Choose a reason for hiding this comment

LukeWood commented Feb 17, 2022

sebastian-sz commented Feb 18, 2022

sebastian-sz commented Mar 7, 2022

bhack commented Mar 7, 2022

LukeWood commented Mar 7, 2022

LukeWood commented Mar 10, 2022

sebastian-sz commented Mar 18, 2022

LukeWood commented Mar 18, 2022

LukeWood commented Mar 18, 2022

LukeWood commented Mar 21, 2022

sebastian-sz commented Mar 22, 2022

LukeWood commented Mar 22, 2022

sebastian-sz commented Mar 23, 2022

LukeWood commented Mar 23, 2022

LukeWood Mar 23, 2022

Choose a reason for hiding this comment

LukeWood commented Mar 23, 2022

bhack commented Mar 23, 2022

LukeWood commented Mar 23, 2022