Skip to content

Releases: innat/UniFormerV2

v1.1

20 Oct 09:39
@innat innat
v1.1
12f615b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
v1.1 Latest
Latest

TensorFlow SavedModel formet weights. Details.

Assets 34

v1.0

18 Oct 11:02
@innat innat
v1.0
9f78c51
Compare
Choose a tag to compare
v1.0

Checkpoints of UniFormerV2 in Keras

Checkpoints of UniFormerV2 model in keras. The pretrained weights are ported from official pytorch model. Following are the list of all available model in .h5 format. The UniFormerV2 checkpoints are available on total 8 datasets, i.e. Kinetics-400/600/700/710, Something Something V2, Moments in Time V1, ActivityNet and HACS. The variants of this models are base and large. Each variants may have further variation for different number of input size and input frame. Please note, the UniFormerV2 large checkpoints for something-something-v2 are above 2 GB, makes it unable to upload here, but it can be found on gdrive. That gives around 35 checkpoints for UniFormerV2.

Checkpoint Naming Style

For the variation and brevity, the general format is:

dataset = 'K400' # K600, K700, ..., SSV2, HACS
dataset_ext = 'K710' # 'K710', 'K710+K400`, None
size = 'B' # 'B', 'L'
patch_size = 16 # 16, 14
num_frames = 32 # 8, 16, 32, 64
input_size = 224 # 224, 336

>> checkpoint_name = (
   f'TFUniFormerV2_'
   f'{dataset}_'
   f'{dataset_ext + "_" if dataset_ext else ""}'
   f'{size}{patch_size.get(size)}_'
   f'{num_frames}x{input_size}.h5'
)
>> checkpoint_name 
TFUniFormerV2_K400_K710_L14_32x336.h5

Here, size represents base (B), and large (L). The dataset_ext refers the initialized pretrained weights while training the model. For all the uniformer-v2 model, the CLIP-400M pretrained weight is used. For being common to all weight, the term is not included in the checkpoint name. For more information about pretrained weight for each checkpoint, see the model zoo page.

Highlights

K400 Usage
  • K400_B16_8x224
  • K400_K710_B16_8x224
  • K400_K710_L14_8x224
  • K400_K710_L14_16x224
  • K400_K710_L14_32x224
  • K400_K710_L14_32x336
  • K400_K710_L14_64x336
 

from uniformerv2 import UniFormerV2
model_name = 'K400_B16_8x224'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 8, 224, 224, 3)))
print(y.shape) # TensorShape([1, 400])
K600 Usage
  • K600_B16_8x224
  • K600_K710_B16_8x224
  • K600_K710_L14_8x224
  • K600_K710_L14_16x224
  • K600_K710_L14_32x224
  • K600_K710_L14_32x336
  • K600_K710_L14_64x336
 

from uniformerv2 import UniFormerV2
model_name = 'K600_B16_8x224'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 8, 224, 224, 3)))
print(y.shape) # TensorShape([1, 600])
K700 Usage
  • K700_B16_8x224
  • K700_K710_B16_8x224
  • K700_K710_L14_8x224
  • K700_K710_L14_16x224
  • K700_K710_L14_32x224
  • K700_K710_L14_32x336
  • K700_K710_L14_64x336
 

from uniformerv2 import UniFormerV2
model_name = 'K700_K710_L14_32x336'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 32, 336, 336, 3)))
print(y.shape) # TensorShape([1, 700])
K710 Usage
  • K710_B16_8x224
  • K710_L14_8x224
  • K710_L14_8x336
 

from uniformerv2 import UniFormerV2
model_name = 'K710_L14_8x336'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 8, 336, 336, 3)))
print(y.shape) # TensorShape([1, 710])
SSV2 Usage
  • SSV2_B16_16x224
  • SSV2_B16_32x224
  • SSV2_L14_16x224
  • SSV2_L14_32x224 

from uniformerv2 import UniFormerV2
model_name = 'SSV2_L14_32x224'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 32, 224, 224, 3)))
print(y.shape) # TensorShape([1, 174])
ActivityNet Usage
  • ANET_L14_16x224
  • ANET_L14_32x224 

from uniformerv2 import UniFormerV2
model_name = 'ANET_L14_16x224'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 16, 224, 224, 3)))
print(y.shape) # TensorShape([1, 200])
HACS Usage
  • HACS_L14_16x224
  • HACS_L14_32x224
 

from uniformerv2 import UniFormerV2
model_name = 'HACS_L14_16x224'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 16, 224, 224, 3)))
print(y.shape) # TensorShape([1, 200])
Moments in Time Usage
  • MITV1_K710_K400_B16_8x224
  • MITV1_K710_K400_L14_8x224
  • MITV1_K710_K400_L14_8x336
 

from uniformerv2 import UniFormerV2
model_name = 'MITV1_K710_K400_L14_8x336'  
uniformer_v2 = UniFormerV2(name=model_name)
uniformer_v2.load_weights(f'TFUniFormerV2_{model_name}.h5')
y = uniformer_v2(np.ones(shape=(1, 8, 336, 336, 3)))
print(y.shape) # TensorShape([1, 339])
Loading