Thanks for your great work and you're so kind to share the code. When I ran the code, I found if I wanted to use the custom model or different pre-training parameters (e.g., ImageNet-21k pretrained ViT-16/B), I needed to convert the model file from jax to tensorflow. And the process is complicated and frustrating. Could you please share your model conversion file (especially ViT model), so that this process would not be that hard? By the way, I found there are 4 extra group of features, i.e, {'cls_embedded', 'encoded_sequence', 'position_embedded_input', 'root_output_with_cls'}. And in the appendix it was wrote "Additionally we use patched and embedded image (i.e. tokenized image input), pre-logits (final CLS-embedding) and logits." What are 'encoded_sequence', 'cls_embedded', 'root_output_with_cls' represent for? I notice all of them have 768 dims, but logits seem should have 1000 dims. Thank you in advance!

Hello Utku, Thanks for this great research The evaluate shell with imagenetr50 works well but when I run the evaluate shell with imagenetvitB16 like python -m head2toe.evaluate --config=head2toe/configs_eval/finetune.py:imagenetvitB16 --config.dataset='data.caltech101' --config.max_num_gpus 1

I got the TypeError :

  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/congyu/TextSum/head2toe/head2toe/evaluate.py", line 148, in <module>
    app.run(main)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/home/congyu/TextSum/head2toe/head2toe/evaluate.py", line 71, in main
    model = finetune_fs_models.FinetuneFS(config)
  File "/home/congyu/TextSum/head2toe/head2toe/models/finetune.py", line 118, in __init__
    res = self.load_backbones()
  File "/home/congyu/TextSum/head2toe/head2toe/models/finetune.py", line 145, in load_backbones
    outputs = backbone(resized_inputs)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/keras/engine/base_layer.py", line 977, in __call__
    input_list)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/keras/engine/base_layer.py", line 1115, in _functional_construction_call
    inputs, input_masks, args, kwargs)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/keras/engine/base_layer.py", line 848, in _keras_tensor_symbolic_call
    return self._infer_output_signature(inputs, args, kwargs, input_masks)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/keras/engine/base_layer.py", line 888, in _infer_output_signature
    outputs = call_fn(inputs, *args, **kwargs)
  File "/home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 695, in wrapper
    raise e.ag_error_metadata.to_exception(e)
TypeError: in user code:

    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow_hub/keras_layer.py:237 call  *
        result = smart_cond.smart_cond(training,
    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/eager/function.py:1707 __call__  **
        return self._call_impl(args, kwargs)
    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/eager/function.py:1723 _call_impl
        raise structured_err
    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/eager/function.py:1717 _call_impl
        cancellation_manager)
    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/eager/function.py:1794 _call_with_structured_signature
        self._structured_signature_check_missing_args(args, kwargs)
    /home/congyu/miniconda3/envs/tensorflow/lib/python3.7/site-packages/tensorflow/python/eager/function.py:1815 _structured_signature_check_missing_args
        ", ".join(sorted(missing_arguments))))

    TypeError: signature_wrapper(*, input_1) missing required arguments: input_1```

Is there something wrong in the vit-pipeline or the shell?

Hello Utku, Thanks for this great research. Just wanted to know if you could publish the instructions to run the codes and possibly training on custom datasets and models. Best regards, Aryan

I'm a little bit confused about which layers are being selected in the Resnet50 Head2Toe implementation. It says in the paper that you select features before and after the final pooling layer. Aren't the features from before the final pooling layer the same as the outputs of the final activation function in the previous Bottleneck block? Wouldn't this result in selecting the same features twice?

Hi! This is some really cool work.

I was hoping if you could clarify some parts of the research paper for me.

How are you able to choose the target size? What I mean is, since the final avgpooling layer in ResNet50 has 2048 input features, does that mean that the maximum number of features you can select from each layer (after pooling to reduce dimensionality) is 2048?

Hi,

Thanks for sharing the code. I wonder if there a reason why setting input range to (-1, 1) (https://github.com/google-research/head2toe/blob/main/head2toe/input_pipeline.py#L152) instead of (0, 1) in vtab(https://github.com/google-research/task_adaptation/blob/master/scripts/run_all_tasks.sh#L57)？

Without the protocol it's treated as a sub directory instead Edit: CLA signed but cannot re-trigger, for such a trivial fix no need to bother please edit directly

./run.sh
+ pip install -r head2toe/requirements.txt
ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'head2toe/requirements.txt'