I updated tensorflow, which updated protocol buffer to version 3.0.0a4. I can't import any model anymore.

To reproduce the bug, go in the caffe-tensorflow/caffe folder, and execute the following code:

python
Python 2.7.10 (default, Jul 13 2015, 12:05:58) 
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from google import protobuf
>>> protobuf.__version__
'3.0.0a4'
>>> import caffepb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "caffepb.py", line 28, in <module>
        type=None),
  File "/Users/olivier/pythonvenv/tf/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 652, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors should not be created directly, but only retrieved from their parent.
>>> exit()

Using the command: ./convert.py VGG_ILSVRC_16_layers_deploy.prototxt --code-output-path=mynet.py

.prototxt obtained from: https://gist.github.com/ksimonyan/211839e770f7b538e2d8

Hello,

I am a beginner to Caffe/TensorFlow/Python. I want to convert existing Caffe's .prototxt to TensorFlow using convert.py, i.e. when running:

root@0352fe53b2be:/src/temporal-segment-networks/lib/caffe-tensorflow-master# ./convert.py /src/temporal-segment-networks/models/ucf101/test1.prototxt --code-output-path=test.py

I get following error:

WARNING: PyCaffe not found!
Falling back to a pure protocol buffer implementation.
* Conversions will be drastically slower.
* This backend is UNTESTED!

Traceback (most recent call last): File "./convert.py", line 60, in main() File "./convert.py", line 56, in main args.phase) File "./convert.py", line 27, in convert transformer = TensorFlowTransformer(def_path, caffemodel_path, phase=phase) File "/src/temporal-segment-networks/lib/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 221, in init self.load(def_path, data_path, phase) File "/src/temporal-segment-networks/lib/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 227, in load graph = GraphBuilder(def_path, phase).build() File "/src/temporal-segment-networks/lib/caffe-tensorflow-master/kaffe/graph.py", line 140, in init self.load() File "/src/temporal-segment-networks/lib/caffe-tensorflow-master/kaffe/graph.py", line 146, in load text_format.Merge(def_file.read(), self.params) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 476, in Merge descriptor_pool=descriptor_pool) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 526, in MergeLines return parser.MergeLines(lines, message) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 559, in MergeLines self._ParseOrMerge(lines, message) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 574, in _ParseOrMerge self._MergeField(tokenizer, message) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 675, in _MergeField merger(tokenizer, message, field) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 764, in _MergeMessageField self._MergeField(tokenizer, sub_message) File "/usr/local/lib/python2.7/dist-packages/google/protobuf/text_format.py", line 643, in _MergeField (message_descriptor.full_name, name)) google.protobuf.text_format.ParseError: 19:3 : Message type "caffe.LayerParameter" has no field named "bn_param". root@0352fe53b2be:/src/temporal-segment-networks/lib/caffe-tensorflow-master# google.protobuf.text_format.ParseError: 19:3 : Message type "caffe.LayerParameter" has no field named "bn_para

Anyone has any ideas about it? Many thanks in advance!

M

I have what I imagine is a rather simple problem. I'm trying to import the standard bvlc_alexnet model from the Caffe github (caffe/models/bvlc_alexnet). I've tried both the train_val and the solver prototext file, and get different errors for each - both in the first layer.

In train_val.prototext, the first layer is:

layer {
  name: "data"
  type: "Data"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 227
    mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
  }
  data_param {
    source: "examples/imagenet/ilsvrc12_train_lmdb"
    batch_size: 256
    backend: LMDB
  }
}

and the error is "Cannot determine the dimensions of data layer". If I use deploy.prototext:

layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }
}

I get an error because the "Input" layer is not recognized, and also because the "input_param" property is not recognized.

I'm sure I'm doing something foolish, but can't seem to figure it out! Perhaps including a sample prototext file that converts successfully would help people like me massage our files into the correct format?

There seems to be an incoherence which prevents me from loading the alexnet weights. I get the following output. As you can see, the output of conv1 is 96 channels, but the input of conv 2 is 48 channels. It does the same with the reference or the alexnet network.

WARNING: PyCaffe not found! 
Falling back to protocol buffer implementation. 
This may take a couple of minutes.
Type                 Name                                          Param               Output
----------------------------------------------------------------------------------------------
Data                 data                                             --    (10, 3, 227, 227)
Convolution          conv1                               (96, 3, 11, 11)     (10, 96, 55, 55)
Pooling              pool1                                            --     (10, 96, 27, 27)
LRN                  norm1                                            --     (10, 96, 27, 27)
Convolution          conv2                               (256, 48, 5, 5)    (10, 256, 27, 27)
Pooling              pool2                                            --    (10, 256, 13, 13)
LRN                  norm2                                            --    (10, 256, 13, 13)
Convolution          conv3                              (384, 256, 3, 3)    (10, 384, 13, 13)
Convolution          conv4                              (384, 192, 3, 3)    (10, 384, 13, 13)
Convolution          conv5                              (256, 192, 3, 3)    (10, 256, 13, 13)
Pooling              pool5                                            --      (10, 256, 6, 6)
InnerProduct         fc6                                    (4096, 9216)     (10, 4096, 1, 1)
InnerProduct         fc7                                    (4096, 4096)     (10, 4096, 1, 1)
InnerProduct         fc8                                    (1000, 4096)     (10, 1000, 1, 1)
Softmax              prob                                             --     (10, 1000, 1, 1)
Converting data...
Saving data...
Saving source...
Done.

I managed to convert my caffe model successfully into 2 files (python class and npy weights file). Now i want to save the converted model as ordinary tensorflow graph.

this is an example for the desired graph file format : [http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz -o /tmp/inceptionv3.tgz](http://download.tensorflow.org/models/image/imagenet/inception-2015-12-05.tgz -o /tmp/inceptionv3.tgz)

Hi, I'm trying to convert a model trained on MNIST in caffe. The conversion to tensorflow is raising an error. The conversion utility is missing the number of channels param for the first Convolution layer. Instead of reading the 4d array as (64, 1, 5, 5) for Conv1 layer, it only recognizes the params as a 3d array viz., (64, 5, 5). Can you pls help me.

Regards

I've been trying to load the standard VGG16 models and prototxt, but am getting the following error when trying to load the very first layer:

Unknown layer type encountered: None

The error is thrown in core.py:GraphBuilder:make_node, which has the following lines:

   kind = NodeKind.map_raw_kind(layer.type)
   if kind is None:
      raise KaffeError('Unknown layer type encountered: %s'%kind)

layer.type here is an integer (4 – no idea where that's coming from), but NodeKind.map_raw_kind is trying to match the layer name (e.g. "Convolution" to LAYER_TYPES, which contains a list of Caffe layer types). Naturally, 4 isn't in LAYER_TYPES, so it throws the error.

I can't figure out where this 4, instead of "Convolution", is coming from – tracing that back led me to kaffe/caffepb, which I don't understand. The layer object looks like this: <kaffe.caffepb.V1LayerParameter object at 0x1064c2a28>, and this is what print(layer) outputs right before the call to layer.type returns 4:

bottom: "data"
top: "conv1_1"
name: "conv1_1"
type: CONVOLUTION
convolution_param {
  num_output: 64
  pad: 1
  kernel_size: 3
}

Any thoughts as to what's going on? I imagine the wrong thing is being assigned to type, but I can't figure out where.

As an aside, the names in LAYER_DESCRIPTORS are all in CamelCase, while the ones in the official prototxts are in all-caps – I've modified manually as a quick fix, but might submit a .lower() fix if I have time today.

Hello,

In kaffe/graph.py:

def make_node(self, layer):
    '''Create a graph node for the giver layer.'''
    kind = NodeKind.map_raw_kind(layer.type)
    if kind is None:
        raise KaffeError('Unknown layer type encountered: %s' % layer.type)
...

layer.type returns number instead of layer name.

As a consequence:

./convert.py --caffemodel model.caffemodel --data-output-path build --code-output-path build deploy.prototxt

fails with:

Error encountered: Unknown layer type encountered: 4 For the first CONVOLUTION layer in deploy.prototxt.

Can anyone help? It seems that protobuf do something wrong. I have 3.0.2 version on my PC.

Hi

I am running TensorFlow 0.11.0rc0 on Python 3.5 with Mac OSX 10.11.6 and libprotoc 2.6.1.

When I try to convert a model (new Caffe format) I get this Protobuf error... Has anyone seen this before?

Traceback (most recent call last): File "convert.py", line 60, in main() File "convert.py", line 56, in main args.phase) File "convert.py", line 27, in convert transformer = TensorFlowTransformer(def_path, caffemodel_path, phase=phase) File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/tensorflow/transformer.py", line 221, in init self.load(def_path, data_path, phase) File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/tensorflow/transformer.py", line 227, in load graph = GraphBuilder(def_path, phase).build() File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/graph.py", line 140, in init self.load() File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/graph.py", line 144, in load self.params = get_caffe_resolver().NetParameter() File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/caffe/resolver.py", line 32, in get_caffe_resolver SHARED_CAFFE_RESOLVER = CaffeResolver() File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/caffe/resolver.py", line 7, in init self.import_caffe() File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/caffe/resolver.py", line 17, in import_caffe from . import caffepb File "/Users/thomaswood/Documents/Clones/caffe-tensorflow/kaffe/caffe/caffepb.py", line 799, in options=_descriptor._ParseOptions(descriptor_pb2.FieldOptions(), '\020\001')), File "/Users/thomaswood/anaconda/envs/py35/lib/python3.5/site-packages/google/protobuf/descriptor.py", line 872, in _ParseOptions message.ParseFromString(string) File "/Users/thomaswood/anaconda/envs/py35/lib/python3.5/site-packages/google/protobuf/message.py", line 185, in ParseFromString self.MergeFromString(serialized) File "/Users/thomaswood/anaconda/envs/py35/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 1088, in MergeFromString raise message_mod.DecodeError('Truncated message.')

I trained GoogleNet (v1) with BatchNorm on Caffe. Prediction is OK on Caffe. When converting to TF model, prediction results are quite random and unpredictable! Then I trained another GoogleNet (v1) without BatchNorm layers on Caffe. Now predictions are OK on both Caffe and the converted TF model. So I think there may be some bugs relevant to BatchNorm part. Have you tested it?

Best,

I am giving deploy_vgg.prototxt as input and followed the instructions given in this site but I get this error: Error encountered: Cannot determine dimensions of data layer. See comments in function shape data for more info. python 3.8 Thank you in advance for your help.

Have tried with python 2 and 3, but keep getting 'multiple top nodes are not supported' then when i remove the top: label from the prototxt it says Layer not found: label

I had an issue loading a model's weights after converting with some error about dtype being object in numpy.load with allow_pickle=False, which must be some sort of environment issue I haven't dug into. Looking into it I see the weights are dumped as a numpy "scalar" of type object, and it's really just a dict of dicts of arrays. hdf5 is a much more natural datatype for this which would be less fragile, and is what keras, tensorflow and scipy use too, so users will have it installed already. Would you be open to a pull request to change this?

Although this library doesn't use keras, keras saves the model architecture in hdf5 too.

2019/12/17 16:50:04 Generic conv implementation does not support grouped convolutions for now. [[{{node conv1/Conv2D}} = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_Placeholder_0_0, conv1/weights)]]

I tried to convert a caffe model, but I got an error. I tried convert the provided example, and I got the same error below.

Python 3.7.4 caffe 1.0.0 tensorflow 1.14.0

$ python convert.py examples/mnist/lenet.prototxt --code-output-path=mynet.py Traceback (most recent call last): File "convert.py", line 60, in main() File "convert.py", line 56, in main args.phase) File "convert.py", line 27, in convert transformer = TensorFlowTransformer(def_path, caffemodel_path, phase=phase) File "/mnt/e/Programming/python/ml/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 221, in init self.load(def_path, data_path, phase) File "/mnt/e/Programming/python/ml/caffe-tensorflow-master/kaffe/tensorflow/transformer.py", line 254, in load print_stderr(self.graph) File "/mnt/e/Programming/python/ml/caffe-tensorflow-master/kaffe/errors.py", line 7, in print_stderr sys.stderr.write('%s\n' % msg) File "/mnt/e/Programming/python/ml/caffe-tensorflow-master/kaffe/graph.py", line 125, in str tuple(out_shape))) TypeError: unsupported format string passed to tuple.format