It is illustrated in figure 2 in origin paper that there are 3 input degrees and 4 output degrees in node operation. In a random graph, we can not guarantee the degree of a node. Is it just a analogy?

https://github.com/seungwonpark/RandWireNN/blob/6bea89c423fe7c0d46b37f4256953e905986c84c/model/dag_layer.py#L39 hi, @seungwonpark : When I read your code, I find that you change(i.e., explode) the output channels in the output node instead of the input node. And I review Kaiming He's paper again, there is an explanation "The channel count in a random graph is increased by 2× when going from one stage to the next stage, following [11]", [11] refers to the Residual Network. So I think we should increase the output channels in the input node. How do you think?

Hello!

I was trying to test some RandWireNNs but did not have enough compute available to prototype and quickly iterate with different random graphs on ImageNet. I decided to add MNIST and CIFAR10 support to allow for much easier experimentation. I suppose this might come in handy to other people trying to do the same.

MNIST comes with standard normalization transfs whereas CIFAR10 is raw (directly from https://www.cs.toronto.edu/~kriz/cifar.html). Additionally, the MNIST loader function directly downloads the dataset to ./data folder.

Hi, I am excited to run your code. I run it on 4 TITAN Xp GPU. And yet, each epoch roughly takes 2 hours. Is that normal? What about your time to train an epoch? Thanks!

Currently, training is done iteratively in while loop and is evaluated every 1000 steps. I think we'd better perform the evaluation at the end of every epoch. I will fix this.

Hello there!

I added support for different datasets in trainer.py and validation.py by adaptively creating the correct dataloader based on hp.data.type in default.yaml and test.yaml. To do so I had to slightly change the dataloader funcs themselves so that they accept the same args as the base ImageNet dataloader function.

I tested the new features on MNIST and as expected it is trivially solved by this model (accuracy not shown here but it was pushing 99% and I only trained for a few thousand iters). The integration is seamless in that nothing is changed in how you call trainer.py from console, although I had to add a new hp.model.input_maps since MNIST images have only 1 input map instead of 3.

I plan on properly testing CIFAR10 next. I'll send you the plots when that is done. MP

From the appendix of paper,

The node indexing strategies for the models are — ER: indices are assigned in a random order; BA: the initial M nodes are assigned indices 1 to M, and all other nodes are indexed following their order of adding to the graph; WS: indices are assigned sequentially in the clockwise order.

The current implementation is assigning random indices for both ER, BA, WS. Since BA/WS generated graphs should not be assigned with random node indices, this is a critical issue. Must be experimented again.

For input nodes of DAGlayer, NodeOp should not multiply input with learnable parameter - it should be passed by itself. I’ll fix it when I become available. (Currently I’m on vacancy for some reason.)

oh, I try to run your code,but I don't have gpu to run ImageNet,so I runned it using tiny-ImageNet which have 200 classes,but it's accuracy up to 30%, And the test loss raised up. Can you help me