Hi, I have created a notebook to showcase fine-tuning using ConvNext-TF on Flower Classification Dataset. It supports both TPU and GPU. Let me know if it checks the requirements.

@sayakpaul I've tried to run conv-next from tf-hub but faced this error. Could you please take a look into this? The error can be reproduced with this script; with following addition:

mdckpt = tf.keras.callbacks.ModelCheckpoint(
    "model.h5", 
    monitor='val_accuracy', 
    verbose=1, 
    save_best_only=True,
    save_weights_only=True, 
    mode='max', 
    save_freq='epoch'
)

Also note that, the error can be addrssed by renaming the layer parameter perhaps? For examle

import uuid

# ref web
def handle_name_exist_issue(model):
    def unique_name():
        return uuid.uuid4().hex.upper()[0:10]

    def postprocess_weight_name(name):
        if len(name.split('/')) == 1:
            return f'{unique_name()}/{name}'
        elif len(name.split('/')) == 2:
            group, name = name.split('/')
            return f'{group}{unique_name()}/{name}'
        elif len(name.split('/')) == 3:
            group, name_1, name_2 = name.split('/')
            return f'{group}{unique_name()}/{name_1}/{name_2}'
   
    model._name = model._name + unique_name()
    for layer in model.layers:
        layer._name = layer._name + unique_name()
    for i in range(len(model.weights)):
        model.weights[i]._handle_name = postprocess_weight_name(model.weights[i].name)
    return model

with strategy.scope(): 
  model = get_model(MODEL_PATH)
  model = handle_name_exist_issue(model)
  model.compile(loss=loss, optimizer=optimizer, metrics=["accuracy"])

history = model.fit(train_dataset, validation_data=val_dataset, 
                   epochs=EPOCHS, callbacks=[mdckpt])

By using handle_name_exist_issue, it solves on Colab (GPU: TF 2.8). But not on Kaggle (TPU TF 2.4.1). I didn't test on Colab TPU and not other TF versions.

Have you faced such issues with these models? I also tried other TF-Hub model, they work as expected. However, any suggestioin for general solutions to work with conv-next-hub models on TPU?

When loading the model in a Kaggle TPU notebook with Tensorflow version 2.4.1 a JSONDecodeError is thrown.

What would be the cause of this error and how could one load a ConvNext model in Tensorflow 2.4.1?

print(f'tensorflow version: {tf.__version__}')
print(f'tensorflow keras version: {tf.keras.__version__}')
print(f'python version: P{sys.version}')
​
model_gcs_path = "gs://tfhub-modules/sayakpaul/convnext_tiny_1k_224/1/uncompressed"
model = tf.keras.models.load_model(model_gcs_path)
print(model.summary(expand_nested=True))

Gives the following error trace:

tensorflow version: 2.4.1
tensorflow keras version: 2.4.0
python version: P3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) 
[GCC 9.3.0]

JSONDecodeError                           Traceback (most recent call last)
/tmp/ipykernel_48/3485362230.py in <module>
      4 
      5 model_gcs_path = "gs://tfhub-modules/sayakpaul/convnext_tiny_1k_224/1/uncompressed"
----> 6 model = tf.keras.models.load_model(model_gcs_path)
      7 print(model.summary(expand_nested=True))

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/save.py in load_model(filepath, custom_objects, compile, options)
    210       if isinstance(filepath, six.string_types):
    211         loader_impl.parse_saved_model(filepath)
--> 212         return saved_model_load.load(filepath, compile, options)
    213 
    214   raise IOError(

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/load.py in load(path, compile, options)
    136   # Recreate layers and metrics using the info stored in the metadata.
    137   keras_loader = KerasObjectLoader(metadata, object_graph_def)
--> 138   keras_loader.load_layers(compile=compile)
    139 
    140   # Generate a dictionary of all loaded nodes.

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/load.py in load_layers(self, compile)
    374       self.loaded_nodes[node_metadata.node_id] = self._load_layer(
    375           node_metadata.node_id, node_metadata.identifier,
--> 376           node_metadata.metadata)
    377 
    378     for node_metadata in metric_list:

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/load.py in _load_layer(self, node_id, identifier, metadata)
    395   def _load_layer(self, node_id, identifier, metadata):
    396     """Load a single layer from a SavedUserObject proto."""
--> 397     metadata = json_utils.decode(metadata)
    398 
    399     # If node was already created

/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/json_utils.py in decode(json_string)
     67 
     68 def decode(json_string):
---> 69   return json.loads(json_string, object_hook=_decode_helper)
     70 
     71 

/opt/conda/lib/python3.7/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    359     if parse_constant is not None:
    360         kw['parse_constant'] = parse_constant
--> 361     return cls(**kw).decode(s)

/opt/conda/lib/python3.7/json/decoder.py in decode(self, s, _w)
    335 
    336         """
--> 337         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    338         end = _w(s, end).end()
    339         if end != len(s):

/opt/conda/lib/python3.7/json/decoder.py in raw_decode(self, s, idx)
    353             obj, end = self.scan_once(s, idx)
    354         except StopIteration as err:
--> 355             raise JSONDecodeError("Expecting value", s, err.value) from None
    356         return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Hi there.

May you provide the arguments for "get_convnext_model" function to produce similar models to the lists here: https://github.com/facebookresearch/ConvNeXt#results-and-pre-trained-models

In particular, I need them to generate "convnext_base_1k_224" and "convnext_xlarge_21k_1k_224".

Hi, would you be interested in sharing a web demo on Huggingface Spaces for ConvNeXt-TF?

It would make this model more accessible as it would allow people to try out the model directly from the browser. Some other recent machine learning model repos have set up Spaces for easy access including Convnext:

github: https://github.com/facebookresearch/ConvNeXt Spaces: https://huggingface.co/spaces/akhaliq/convnext

github: https://github.com/salesforce/BLIP Spaces: https://huggingface.co/spaces/akhaliq/BLIP

github: https://github.com/facebookresearch/omnivore Spaces: https://huggingface.co/spaces/akhaliq/omnivore

Spaces is completely free, and I can help setup a Gradio Space. Here are some getting started instructions if you'd prefer to do it yourself: https://huggingface.co/blog/gradio-spaces

• Added Conv2D initializations (https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py#L103)
• Added Dense initializations (https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py#L103)
• Attempted a fix for Padding on the DWConv in the ConvNext block (https://github.com/facebookresearch/ConvNeXt/blob/main/models/convnext.py#L28)